vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #358
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.3MDwu9KUI8', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: b34552b0-ab6c-4f87-850f-02c8ddf5425916gitlab # Superblock backups stored on blocks:17gitlab # 32768, 98304, 163840, 22937618gitlab # 19(finished: start all VMs, in 69.97 seconds)20gitlab # Allocating group tables: 0/8 done21gitlab: waiting for unit setup-gitlab.service22gitlab # Writing inode tables: 0/8 done23gitlab: waiting for the VM to finish booting24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9mrn48hngamshgk3fiwc2l4h4xrrj7ph-nixos-system-gitlab-test/init regInfo=/nix/store/rmbsrvdh842qpf07k6pd296j15701bix-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/rmbsrvdh842qpf07k6pd296j15701bix-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000028] arm-pv: using stolen time PV108gitlab # [ 0.000358] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000546] Console: colour dummy device 80x25110gitlab # [ 0.000553] printk: legacy console [tty0] enabled111gitlab # [ 0.000737] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000744] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000830] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000969] landlock: Up and running.115gitlab # [ 0.000972] Yama: becoming mindful.116gitlab # [ 0.001406] LSM support for eBPF active117gitlab # [ 0.001574] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001686] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002833] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003494] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003498] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003646] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004644] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.004737] EFI services will not be available.125gitlab # [ 0.004920] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005587] Detected PIPT I-cache on CPU1127gitlab # [ 0.005687] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005822] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.005951] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007055] Detected PIPT I-cache on CPU2131gitlab # [ 0.007164] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007295] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007424] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008495] Detected PIPT I-cache on CPU3135gitlab # [ 0.008615] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008747] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.008882] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.009425] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.009451] SMP: Total of 4 processors activated.140gitlab # [ 0.009454] CPU: All CPU(s) started at EL1141gitlab # [ 0.009462] CPU features: detected: Branch Target Identification142gitlab # [ 0.009466] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.009469] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.009473] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.009476] CPU features: detected: Common not Private translations146gitlab # [ 0.009479] CPU features: detected: CRC32 instructions147gitlab # [ 0.009481] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.009484] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.009488] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.009490] CPU features: detected: E0PD151gitlab # [ 0.009492] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.009495] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.009498] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.009501] CPU features: detected: Fine Grained Traps155gitlab # [ 0.009504] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.009508] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.009510] CPU features: detected: LSE atomic instructions158gitlab # [ 0.009513] CPU features: detected: Privileged Access Never159gitlab # [ 0.009515] CPU features: detected: PMUv3160gitlab # [ 0.009518] CPU features: detected: RAS Extension Support161gitlab # [ 0.009520] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.009523] CPU features: detected: Random Number Generator163gitlab # [ 0.009525] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.009528] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.009530] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.009534] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.009661] alternatives: applying system-wide alternatives168gitlab # [ 0.012587] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.012827] Memory: 5963752K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.014333] devtmpfs: initialized171gitlab # [ 0.018387] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.018458] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.018654] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.018659] 0 pages in range for non-PLT usage175gitlab # [ 0.018660] 508288 pages in range for PLT usage176gitlab # [ 0.018747] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.019534] DMI not present or invalid.178gitlab # [ 0.022796] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.025516] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.025807] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.027654] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.027676] audit: initializing netlink subsys (disabled)183gitlab # [ 0.027978] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.028462] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.028465] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.028469] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.028472] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.028474] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.028566] cpuidle: using governor ladder190gitlab # [ 0.028590] cpuidle: using governor menu191gitlab # [ 0.028812] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.029050] ASID allocator initialised with 65536 entries193gitlab # [ 0.030172] Serial: AMBA PL011 UART driver194gitlab # [ 0.035359] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.035692] printk: console [ttyAMA0] enabled196gitlab # [ 0.047276] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.047297] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.047307] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.047315] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.047324] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.047332] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.047340] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.047348] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.059666] fbcon: Taking over console205gitlab # [ 0.059677] ACPI: Interpreter disabled.206gitlab # [ 0.060661] iommu: Default domain type: Translated207gitlab # [ 0.060677] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.076644] SCSI subsystem initialized209gitlab # [ 0.078859] usbcore: registered new interface driver usbfs210gitlab # [ 0.079006] usbcore: registered new interface driver hub211gitlab # [ 0.079116] usbcore: registered new device driver usb212gitlab # [ 0.080056] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.080067] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.080093] PTP clock support registered215gitlab # [ 0.080204] EDAC MC: Ver: 3.0.0216gitlab # [ 0.081216] scmi_core: SCMI protocol bus registered217gitlab # [ 0.082678] FPGA manager framework218gitlab # [ 0.085247] vgaarb: loaded219gitlab # [ 0.088868] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.090358] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.090422] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.093179] netfs: FS-Cache loaded223gitlab # [ 0.093451] pnp: PnP ACPI: disabled224gitlab # [ 0.104821] NET: Registered PF_INET protocol family225gitlab # [ 0.107486] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.166317] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.166393] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.166456] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.166834] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.167738] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.167890] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.167963] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.168461] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.168746] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.168778] NET: Registered PF_XDP protocol family236gitlab # [ 0.168799] PCI: CLS 0 bytes, default 64237gitlab # [ 0.169289] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.181027] kvm [1]: HYP mode not available239gitlab # [ 0.250744] Initialise system trusted keyrings240gitlab # [ 0.251066] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.251772] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.251920] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.267141] Key type asymmetric registered244gitlab # [ 0.267154] Asymmetric key parser 'x509' registered245gitlab # [ 0.267215] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.267434] io scheduler mq-deadline registered247gitlab # [ 0.267440] io scheduler kyber registered248gitlab # [ 0.278767] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.280906] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.281449] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.281468] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.281496] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.281506] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.281531] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.281560] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.281711] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.281721] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.281727] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.281731] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.281736] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.281893] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.282696] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.283033] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.283051] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.283093] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.283111] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.283965] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.284269] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.284287] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.284326] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.285151] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.285485] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.285503] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.285544] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.286353] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.286679] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.286698] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.286746] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.287577] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.287911] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.287929] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.287972] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.288809] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.289149] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.289168] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.289217] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.290021] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.290327] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.290344] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.290388] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.290405] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.291194] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.291530] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.291571] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.292363] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.292698] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.292738] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.293426] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.293740] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.294136] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.294558] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.294577] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.294618] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.295412] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.295738] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.295756] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.295796] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.296761] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.296770] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.296776] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.296850] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.296935] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.297002] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.297074] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.297143] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.297216] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.297290] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.297358] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.297439] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.297534] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.297601] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.297627] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.297654] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.297682] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.297708] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.297735] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.297762] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.297789] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.297816] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.297843] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.297870] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.297898] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.297925] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.297959] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.297985] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.298010] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.298036] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.298063] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.298089] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.298121] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.298148] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.298180] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.298188] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.298192] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.300121] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.316593] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.319576] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.322378] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.325260] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.330887] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.333653] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.336301] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.339059] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.345618] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.348213] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.351442] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.359008] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.361478] msm_serial: driver initialized359gitlab # [ 0.361723] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.361794] STM32 USART driver initialized361gitlab # [ 0.397940] loop: module loaded362gitlab # [ 0.398625] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.404840] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.409582] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.414686] virtio_blk virtio5: [vdb] 12285000 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.420060] megasas: 07.734.00.00-rc1367gitlab # [ 0.422248] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.429606] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.429809] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.435185] Using buffer write method371gitlab # [ 0.435254] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.439500] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.439534] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.444809] Using buffer write method375gitlab # [ 0.444845] Concatenating MTD devices:376gitlab # [ 0.444851] (0): "0.flash"377gitlab # [ 0.444899] (1): "0.flash"378gitlab # [ 0.444906] into device "0.flash"379gitlab # [ 0.693390] Freeing initrd memory: 27184K380gitlab # [ 0.712725] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.721959] thunder_xcv, ver 1.0382gitlab # [ 0.722057] thunder_bgx, ver 1.0383gitlab # [ 0.722128] nicpf, ver 1.0384gitlab # [ 0.723815] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.723827] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.723896] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.723911] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.723984] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.723994] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.724095] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.724106] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.724492] sky2: driver version 1.30393gitlab # [ 0.729572] usbcore: registered new interface driver usb-storage394gitlab # [ 0.729847] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.729918] ehci-pci 0000:00:0a.0: EHCI Host Controller396gitlab # [ 0.729939] usbserial: USB Serial support registered for generic397gitlab # [ 0.729982] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.730581] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.731739] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.734567] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.734631] rtc-pl031 9010000.pl031: setting system clock to 2026-09-08T18:36:01 UTC (1788892561)402gitlab # [ 0.735606] i2c_dev: i2c /dev entries driver403gitlab # [ 0.740982] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00404gitlab # [ 0.742054] hub 1-0:1.0: USB hub found405gitlab # [ 0.742159] hub 1-0:1.0: 6 ports detected406gitlab # [ 0.743976] sdhci: Secure Digital Host Controller Interface driver407gitlab # [ 0.743990] sdhci: Copyright(c) Pierre Ossman408gitlab # [ 0.744779] Synopsys Designware Multimedia Card Interface Driver409gitlab # [ 0.745947] sdhci-pltfm: SDHCI platform and OF driver helper410gitlab # [ 0.750853] hid: raw HID events driver (C) Jiri Kosina411gitlab # [ 0.751857] usbcore: registered new interface driver usbhid412gitlab # [ 0.751868] usbhid: USB HID core driver413gitlab # [ 0.794048] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.798947] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.799401] NET: Registered PF_INET6 protocol family416gitlab # [ 0.801744] Segment Routing with IPv6417gitlab # [ 0.801788] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.801916] NET: Registered PF_PACKET protocol family419gitlab # [ 0.802303] 9pnet: Installing 9P2000 support420gitlab # [ 0.807121] Key type dns_resolver registered421gitlab # [ 0.826538] registered taskstats version 1422gitlab # [ 0.827293] Loading compiled-in X.509 certificates423gitlab # [ 0.854638] Demotion targets for Node 0: null424gitlab # [ 0.854888] Key type .fscrypt registered425gitlab # [ 0.854897] Key type fscrypt-provisioning registered426gitlab # [ 0.855106] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.855152] ima: Allocated hash algorithm: sha1428gitlab # [ 0.855220] ima: No architecture policies found429gitlab # [ 0.856658] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.892220] clk: Disabling unused clocks431gitlab # [ 0.892254] PM: genpd: Disabling unused power domains432gitlab # [ 0.897999] Freeing unused kernel memory: 4736K433gitlab # [ 0.898261] Run /init as init process434gitlab # [ 0.944754] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.988942] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.143761] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.229720] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0438gitlab # [ 1.279760] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)439gitlab # [ 1.279828] systemd[1]: Detected virtualization qemu.440gitlab # [ 1.279904] systemd[1]: Detected architecture arm64.441gitlab # [ 1.279920] systemd[1]: Running in initrd.442gitlab # [ 1.280804] systemd[1]: Initializing machine ID from random generator.443gitlab # [ 1.281180] systemd[1]: Hostname set to <gitlab>.444gitlab # [ 1.353139] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.507363] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2446gitlab # [ 1.507770] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0447gitlab # [ 1.663018] systemd[1]: bpf-restrict-fs: LSM BPF program attached448gitlab # [ 1.771384] systemd[1]: Queued start job for default target Initrd Default Target.449gitlab # [ 1.793524] systemd[1]: Created slice Slice /system/modprobe.450gitlab # [ 1.793917] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.451gitlab # [ 1.793976] systemd[1]: Expecting device /dev/disk/by-label/nix-store...452gitlab # [ 1.793997] systemd[1]: Expecting device /dev/disk/by-label/nixos...453gitlab # [ 1.794037] systemd[1]: Reached target Path Units.454gitlab # [ 1.794063] systemd[1]: Reached target Slice Units.455gitlab # [ 1.794090] systemd[1]: Reached target Swaps.456gitlab # [ 1.794120] systemd[1]: Reached target Timer Units.457gitlab # [ 1.794445] systemd[1]: Listening on D-Bus System Message Bus Socket.458gitlab # [ 1.794790] systemd[1]: Listening on Journal Socket (/dev/log).459gitlab # [ 1.795125] systemd[1]: Listening on Journal Sockets.460gitlab # [ 1.795365] systemd[1]: Listening on udev Control Socket.461gitlab # [ 1.795501] systemd[1]: Listening on udev Kernel Socket.462gitlab # [ 1.795532] systemd[1]: Reached target Socket Units.463gitlab # [ 1.798182] systemd[1]: Starting Create List of Static Device Nodes...464gitlab # [ 1.800849] systemd[1]: Starting Load Kernel Module 9pnet_virtio...465gitlab # [ 1.800994] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs466gitlab # [ 1.803885] systemd[1]: Mounting Kernel Configuration File System...467gitlab # [ 1.833179] systemd[1]: Starting Journal Service...468gitlab # [ 1.835980] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.836152] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.838211] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.845085] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.846798] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.849343] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.850638] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.858800] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...476gitlab # [ 1.884753] systemd-journald[95]: Collecting audit messages is disabled.477gitlab # [ 1.888411] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0478gitlab # [ 1.888694] [drm] features: -virgl +edid -resource_blob -host_visible479gitlab # [ 1.888700] [drm] features: -context_init480gitlab # [ 1.890090] [drm] number of scanouts: 1481gitlab # [ 1.890113] [drm] number of cap sets: 0482gitlab # [ 1.897303] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.483gitlab # [ 1.899664] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic484gitlab # [ 1.899704] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0485gitlab # [ 1.905119] systemd[1]: Starting Create Static Device Nodes in /dev...486gitlab # [ 1.905362] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.487gitlab # [ 1.908978] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev488gitlab # [ 1.909891] Console: switching to colour frame buffer device 160x50489gitlab # [ 1.916672] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device490gitlab # [ 1.924151] systemd[1]: Finished Create Static Device Nodes in /dev.491gitlab # [ 1.924608] systemd[1]: Reached target Preparation for Local File Systems.492gitlab # [ 1.925171] systemd[1]: Reached target Local File Systems.493gitlab # [ 1.937196] systemd[1]: Starting Rule-based Manager for Device Events and Files...494gitlab # [ 1.943262] systemd[1]: Finished Load Kernel Modules.495gitlab # [ 1.942223] systemd-modules-load[96]: Using 4 probe threads496gitlab # [ 1.943576] systemd-modules-load[96]: Module 'virtio_balloon' is built in497gitlab # [ 1.944909] systemd-modules-load[96]: Module 'virtio_console' is built in498gitlab # [ 1.946061] systemd-modules-load[96]: Module 'virtio_rng' is built in499gitlab # [ 1.947127] systemd-modules-load[96]: Inserted module 'dm_mod'500gitlab # [ 1.948138] systemd-modules-load[96]: Inserted module 'virtio_gpu'[ 1.953226] systemd[1]: Starting Apply Kernel Variables...501gitlab # 502gitlab # [ 1.956511] systemd[1]: Started Journal Service.503gitlab # [ 1.965147] systemd[1]: Starting Create System Files and Directories...504gitlab # [ 1.976178] systemd[1]: Finished Apply Kernel Variables.505gitlab # [ 1.978093] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.980544] systemd-udevd[105]: Using default interface naming scheme 'v261'.507gitlab # [ 2.004926] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 2.028852] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 2.093015] systemd-vconsole-setup[128]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 2.098242] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.508488] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.509784] systemd[1]: Reached target System Initialization.513gitlab # [ 2.510628] systemd[1]: Reached target Basic System.514gitlab # [ 2.700405] (udev-worker)[123]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.705693] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.712737] (udev-worker)[119]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.716885] systemd[1]: Found device /dev/disk/by-label/nix-store.518gitlab # [ 2.774432] systemd[1]: Found device /dev/disk/by-label/nixos.519gitlab # [ 2.777557] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.779496] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.829980] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.834583] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.837445] systemd[1]: Mounting /sysroot...524gitlab # [ 2.882276] EXT4-fs (vda): mounted filesystem b34552b0-ab6c-4f87-850f-02c8ddf54259 r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.880419] systemd[1]: Mounted /sysroot.526gitlab # [ 2.882747] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.888185] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.889258] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.893563] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.895354] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.915543] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.917385] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.920881] systemd[1]: Mounted /sysroot/run.534gitlab # [ 2.934459] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload.535gitlab # [ 2.941529] systemd[1]: Mounted /sysroot/nix/.rw-store.536gitlab # [ 2.957424] systemd[1]: Starting rw-sysroot-nix-store.service...537gitlab # [ 2.959357] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)...538gitlab # [ 2.961134] systemd[1]: Reloading...539gitlab # [ 3.200746] erofs (device vdb): mounted with root inode @ nid 128.540gitlab # [ 3.255096] systemd[1]: Reloading finished in 295 ms.541gitlab # [ 3.303445] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace...542gitlab # [ 3.305291] systemd[1]: Mounted /sysroot/nix/.ro-store.543gitlab # [ 3.306751] systemd[1]: Mounted /sysroot/tmp/shared.544gitlab # [ 3.308190] systemd[1]: Mounted /sysroot/tmp/xchg.545gitlab # [ 3.308945] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.546gitlab # [ 3.310512] systemd[1]: Finished rw-sysroot-nix-store.service.547gitlab # [ 3.311880] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace...548gitlab # [ 3.316819] systemd[1]: initrd-parse-etc.service: Deactivated successfully.549gitlab # [ 3.319650] systemd[1]: Finished Mountpoints Configured in the Real Root.550gitlab # [ 3.320949] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.551gitlab # [ 3.322027] systemd[1]: Starting rw-sysroot-nix-store.service...552gitlab # [ 3.348933] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.553gitlab # [ 3.350602] systemd[1]: Finished rw-sysroot-nix-store.service.554gitlab # [ 3.379939] (udev-worker)[143]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory555gitlab # [ 3.382955] (udev-worker)[143]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory556gitlab # [ 3.393532] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.557gitlab # [ 3.395267] systemd[1]: Stopped Virtual Console Setup.558gitlab # [ 3.396273] systemd[1]: Stopping Virtual Console Setup...559gitlab # [ 3.397069] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 3.424945] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.561gitlab # [ 3.426188] systemd[1]: Stopped Virtual Console Setup.562gitlab # [ 3.427032] systemd[1]: Starting Virtual Console Setup...563gitlab # [ 3.451416] systemd-vconsole-setup[202]: Configuration of first virtual console was skipped, ignoring remaining ones.564gitlab # [ 3.454510] systemd[1]: Finished Virtual Console Setup.565gitlab # [ 3.842261] systemd[1]: Mounting /sysroot/nix/store...566gitlab # [ 3.910410] systemd[1]: Mounted /sysroot/nix/store.567gitlab # [ 3.912734] systemd[1]: Reached target Initrd File Systems.568gitlab # [ 3.915162] systemd[1]: Starting Find NixOS closure...569gitlab # [ 3.917669] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...570gitlab # [ 3.964649] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.571gitlab # [ 3.968333] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.572gitlab # [ 3.985838] systemd[1]: Finished Find NixOS closure.573gitlab # [ 3.988306] systemd[1]: Reached target Initrd Default Target.574gitlab # [ 3.990814] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...575gitlab # [ 4.043788] systemd[1]: Stopped target Initrd Default Target.576gitlab # [ 4.046518] systemd[1]: Stopped target Basic System.577gitlab # [ 4.048819] systemd[1]: Stopped target Initrd Root Device.578gitlab # [ 4.051216] systemd[1]: Stopped target Path Units.579gitlab # [ 4.053457] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.580gitlab # [ 4.056848] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.581gitlab # [ 4.060115] systemd[1]: Stopped target Slice Units.582gitlab # [ 4.062296] systemd[1]: Stopped target Socket Units.583gitlab # [ 4.067406] systemd[1]: Stopped target System Initialization.584gitlab # [ 4.071327] systemd[1]: Stopped target Swaps.585gitlab # [ 4.073627] systemd[1]: Stopped target Timer Units.586gitlab # [ 4.075800] systemd[1]: dbus.socket: Deactivated successfully.587gitlab # [ 4.079432] systemd[1]: Closed D-Bus System Message Bus Socket.588gitlab # [ 4.082476] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.589gitlab # [ 4.086442] systemd[1]: Stopped Find NixOS closure.590gitlab # [ 4.089707] systemd[1]: Starting Load Kernel Module 9pnet_virtio...591gitlab # [ 4.092596] systemd[1]: Starting rw-sysroot-nix-store.service...592gitlab # [ 4.095400] systemd[1]: systemd-sysctl.service: Deactivated successfully.593gitlab # [ 4.098363] systemd[1]: Stopped Apply Kernel Variables.594gitlab # [ 4.100652] systemd[1]: systemd-modules-load.service: Deactivated successfully.595gitlab # [ 4.103261] systemd[1]: Stopped Load Kernel Modules.596gitlab # [ 4.105207] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.597gitlab # [ 4.107862] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.598gitlab # [ 4.110423] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.599gitlab # [ 4.112836] systemd[1]: Stopped Create System Files and Directories.600gitlab # [ 4.114805] systemd[1]: Stopped target Local File Systems.601gitlab # [ 4.116542] systemd[1]: Stopped target Preparation for Local File Systems.602gitlab # [ 4.118550] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.603gitlab # [ 4.120686] systemd[1]: Stopped Coldplug All udev Devices.604gitlab # [ 4.122181] systemd[1]: Stopping Rule-based Manager for Device Events and Files...605gitlab # [ 4.124190] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.606gitlab # [ 4.125996] systemd[1]: Stopped Virtual Console Setup.607gitlab # [ 4.127305] systemd[1]: initrd-cleanup.service: Deactivated successfully.608gitlab # [ 4.129019] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.609gitlab # [ 4.130605] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.610gitlab # [ 4.132335] systemd[1]: Finished rw-sysroot-nix-store.service.611gitlab # [ 4.133634] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.612gitlab # [ 4.135217] systemd[1]: Finished Load Kernel Module 9pnet_virtio.613gitlab # [ 4.136614] systemd[1]: systemd-udevd.service: Deactivated successfully.614gitlab # [ 4.138156] systemd[1]: Stopped Rule-based Manager for Device Events and Files.615gitlab # [ 4.139655] systemd[1]: systemd-udevd.service: Consumed 2.418s CPU time over 2.184s wall clock time, 31.1M memory peak.616gitlab # [ 4.141900] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.617gitlab # [ 4.143416] systemd[1]: Closed udev Control Socket.618gitlab # [ 4.144536] systemd[1]: Starting Cleanup udev Database...619gitlab # [ 4.145759] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.620gitlab # [ 4.147542] systemd[1]: Stopped Create Static Device Nodes in /dev.621gitlab # [ 4.148971] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.622gitlab # [ 4.150671] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.623gitlab # [ 4.152302] systemd[1]: kmod-static-nodes.service: Deactivated successfully.624gitlab # [ 4.154765] systemd[1]: Stopped Create List of Static Device Nodes.625gitlab # [ 4.222532] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.626gitlab # [ 4.225832] systemd[1]: Finished Cleanup udev Database.627gitlab # [ 4.228087] systemd[1]: Reached target Switch Root.628gitlab # [ 4.230249] systemd[1]: Starting NixOS Activation...629gitlab # [ 4.412824] initrd-nixos-activation-start[227]: booting system configuration /nix/store/9mrn48hngamshgk3fiwc2l4h4xrrj7ph-nixos-system-gitlab-test630gitlab # [ 4.475368] initrd-nixos-activation-start[227]: running activation script...631gitlab # [ 4.811823] initrd-nixos-activation-start[251]: setting up /etc...632gitlab # [ 4.987798] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.633gitlab # [ 4.991368] systemd[1]: Finished NixOS Activation.634gitlab # [ 4.993637] systemd[1]: Starting Switch Root...635gitlab # [ 5.039700] systemd[1]: Switching root.636gitlab # [ 5.189316] systemd-journald[95]: Received SIGTERM from PID 1 (systemd).637gitlab # [ 5.846072] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)638gitlab # [ 5.846212] systemd[1]: Detected virtualization qemu.639gitlab # [ 5.846284] systemd[1]: Detected architecture arm64.640gitlab # [ 5.846454] systemd[1]: Detected first boot.641gitlab # [ 5.847806] systemd[1]: Initializing machine ID from random generator.642gitlab # [ 6.078713] systemd[1]: bpf-restrict-fs: LSM BPF program attached643gitlab # [ 6.183712] systemd[1]: Applying preset policy.644gitlab # [ 6.369736] systemd[1]: Populated /etc with preset unit settings.645gitlab # [ 6.523727] systemd[1]: initrd-switch-root.service: Deactivated successfully.646gitlab # [ 6.524177] systemd[1]: Stopped initrd-switch-root.service.647gitlab # [ 6.526551] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.648gitlab # [ 6.531075] systemd[1]: Created slice Slice /system/getty.649gitlab # [ 6.537180] systemd[1]: Created slice GitLab DevOps Platform Slice.650gitlab # [ 6.540536] systemd[1]: Created slice User and Session Slice.651gitlab # [ 6.544032] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.652gitlab # [ 6.545238] systemd[1]: Started Forward Password Requests to Wall Directory Watch.653gitlab # [ 6.545972] systemd[1]: Expecting device /dev/hvc0...654gitlab # [ 6.546628] systemd[1]: Expecting device /dev/ttyAMA0...655gitlab # [ 6.547280] systemd[1]: Reached target Local Encrypted Volumes.656gitlab # [ 6.547933] systemd[1]: Stopped target initrd-fs.target.657gitlab # [ 6.548569] systemd[1]: Stopped target initrd-root-fs.target.658gitlab # [ 6.549469] systemd[1]: Stopped target initrd-switch-root.target.659gitlab # [ 6.550487] systemd[1]: Reached target Virtual Machines and Containers.660gitlab # [ 6.551563] systemd[1]: Reached target Path Units.661gitlab # [ 6.552625] systemd[1]: Reached target Remote File Systems.662gitlab # [ 6.553702] systemd[1]: Reached target Slice Units.663gitlab # [ 6.554740] systemd[1]: Reached target Swaps.664gitlab # [ 6.558630] systemd[1]: Listening on Query the User Interactively for a Password.665gitlab # [ 6.562647] systemd[1]: Listening on Process Core Dump Socket.666gitlab # [ 6.565696] systemd[1]: Listening on Credential Encryption/Decryption.667gitlab # [ 6.569206] systemd[1]: Listening on Factory Reset Management.668gitlab # [ 6.570536] systemd[1]: Listening on Hostname Service Socket.669gitlab # [ 6.579959] systemd[1]: Starting Journal Log Access Socket...670gitlab # [ 6.582011] systemd[1]: Listening on Journal Audit Socket.671gitlab # [ 6.584730] systemd[1]: Listening on Console Output Muting Service Socket.672gitlab # [ 6.585818] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.673gitlab # [ 6.586607] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os674gitlab # [ 6.587319] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki675gitlab # [ 6.591246] systemd[1]: Listening on Disk Repartitioning Service Socket.676gitlab # [ 6.592140] systemd[1]: Listening on udev Control Socket.677gitlab # [ 6.593083] systemd[1]: Listening on udev Varlink Socket.678gitlab # [ 6.601103] systemd[1]: Mounting Huge Pages File System...679gitlab # [ 6.608168] systemd[1]: Mounting POSIX Message Queue File System...680gitlab # [ 6.621871] systemd[1]: Mounting Kernel Debug File System...681gitlab # [ 6.628078] systemd[1]: Mounting Kernel Trace File System...682gitlab # [ 6.638219] systemd[1]: Starting Create List of Static Device Nodes...683gitlab # [ 6.657272] systemd[1]: Starting Load Kernel Module 9pnet_virtio...684gitlab # [ 6.657988] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs685gitlab # [ 6.669057] systemd[1]: Mounting Kernel Configuration File System...686gitlab # [ 6.669673] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm687gitlab # [ 6.670492] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore688gitlab # [ 6.678207] systemd[1]: Starting Load Kernel Module fuse...689gitlab # [ 6.678803] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67690gitlab # [ 6.691917] systemd[1]: Starting Journal Service...691gitlab # [ 6.695842] systemd[1]: Starting Load Kernel Modules...692gitlab # [ 6.705290] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...693gitlab # [ 6.713353] systemd[1]: Starting Remount Root and Kernel File Systems...694gitlab # [ 6.714142] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os695gitlab # [ 6.725332] fuse: init (API version 7.45)696gitlab # [ 6.729190] systemd[1]: Starting Coldplug All udev Devices...697gitlab # [ 6.737338] systemd[1]: Listening on Journal Log Access Socket.698gitlab # [ 6.738600] systemd[1]: Mounted Huge Pages File System.699gitlab # [ 6.739963] systemd[1]: Mounted POSIX Message Queue File System.700gitlab # [ 6.741445] systemd[1]: Mounted Kernel Debug File System.701gitlab # [ 6.745073] systemd[1]: Mounted Kernel Trace File System.702gitlab # [ 6.746835] systemd[1]: Finished Create List of Static Device Nodes.703gitlab # [ 6.748276] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.704gitlab # [ 6.749593] systemd[1]: Finished Load Kernel Module 9pnet_virtio.705gitlab # [ 6.750724] systemd[1]: Mounted Kernel Configuration File System.706gitlab # [ 6.751994] systemd[1]: modprobe@fuse.service: Deactivated successfully.707gitlab # [ 6.753329] systemd[1]: Finished Load Kernel Module fuse.708gitlab # [ 6.754776] systemd[1]: Finished Load Kernel Modules.709gitlab # [ 6.757956] EXT4-fs (vda): re-mounted b34552b0-ab6c-4f87-850f-02c8ddf54259.710gitlab # [ 6.760481] systemd[1]: Mounting FUSE Control File System...711gitlab # [ 6.769464] systemd[1]: Starting Firewall...712gitlab # [ 6.773288] systemd[1]: Starting Apply Kernel Variables...713gitlab # [ 6.778277] systemd-journald[322]: Collecting audit messages is enabled.714gitlab # [ 6.781072] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...715gitlab # [ 6.788692] systemd[1]: Finished Remount Root and Kernel File Systems.716gitlab # [ 6.791502] systemd[1]: Mounted FUSE Control File System.717gitlab # [ 6.797437] systemd[1]: Listening on Disk Image Download Service Socket.718gitlab # [ 6.798286] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore719gitlab # [ 6.797337] systemd[1]: Queued start job for default target Multi-User System.720gitlab # [ 6.803336] systemd[1]: Starting Load/Save OS Random Seed...721gitlab # [ 6.805643] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os722gitlab # [ 6.806512] systemd[1]: Started Journal Service.723gitlab # [ 6.801645] systemd[1]: Unnecessary job was removed for /dev/vdb.724gitlab # [ 6.806786] systemd[1]: systemd-journald.service: Deactivated successfully.725gitlab # [ 6.810325] systemd-modules-load[323]: Using 3 probe threads726gitlab # [ 6.812764] systemd-modules-load[323]: Module 'atkbd' is built in727gitlab # [ 6.817489] systemd-modules-load[323]: Module 'loop' is built in728gitlab # [ 6.820407] systemd-modules-load[323]: Inserted module 'tls'729gitlab # [ 6.822168] systemd-oomd[324]: No swap; memory pressure usage will be degraded730gitlab # [ 6.827431] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.731gitlab # [ 6.829207] systemd[1]: Starting Flush Journal to Persistent Storage...732gitlab # [ 6.830473] systemd[1]: Finished Apply Kernel Variables.733gitlab # [ 6.836752] systemd-journald[322]: Received client request to flush runtime journal.734gitlab # [ 6.862692] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.735gitlab # [ 6.864165] systemd[1]: Finished Load/Save OS Random Seed.736gitlab # [ 6.865294] systemd[1]: Reached target First Boot Complete.737gitlab # [ 6.866383] systemd[1]: Starting Create Static Device Nodes in /dev...738gitlab # [ 6.870075] systemd[1]: Finished Flush Journal to Persistent Storage.739gitlab # [ 6.878421] systemd[1]: Finished Create Static Device Nodes in /dev.740gitlab # [ 6.879663] systemd[1]: Reached target Preparation for Local File Systems.741gitlab # [ 6.882455] systemd[1]: Starting Rule-based Manager for Device Events and Files...742gitlab # [ 6.953822] systemd-udevd[373]: Using default interface naming scheme 'v261'.743gitlab # [ 6.994809] systemd[1]: Started Rule-based Manager for Device Events and Files.744gitlab # [ 7.209663] systemd[1]: Finished Firewall.745gitlab # [ 7.410567] systemd[1]: Finished Coldplug All udev Devices.746gitlab # [ 7.487989] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs747gitlab # [ 7.562590] systemd[1]: Mounting /run/wrappers...748gitlab # [ 7.571073] (udev-worker)[413]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.749gitlab # [ 7.578571] (udev-worker)[413]: Network interface NamePolicy= disabled on kernel command line.750gitlab # [ 7.589227] (udev-worker)[479]: Network interface NamePolicy= disabled on kernel command line.751gitlab # [ 7.594405] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.752gitlab # [ 7.608853] systemd[1]: Mounted /run/wrappers.753gitlab # [ 7.612289] systemd[1]: Reached target Local File Systems.754gitlab # [ 7.613304] systemd[1]: Listening on Boot Loader Control Service Socket.755gitlab # [ 7.619629] systemd[1]: Starting register-nix-paths.service...756gitlab # [ 7.621381] systemd[1]: Starting Create SUID/SGID Wrappers...757gitlab # [ 7.623434] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.758gitlab # [ 7.640151] systemd[1]: Starting Save Transient machine-id to Disk...759gitlab # [ 7.652273] systemd[1]: Starting Create System Files and Directories...760gitlab # [ 7.673453] systemd[1]: Condition check resulted in Virtio network device being skipped.761gitlab # [ 7.680466] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.762gitlab # [ 7.686803] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore763gitlab # [ 7.691914] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.764gitlab # [ 7.693955] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67765gitlab # [ 7.696736] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore766gitlab # [ 7.698848] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os767gitlab # [ 7.700595] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os768gitlab # [ 7.702168] systemd[1]: Finished Create System Files and Directories.769gitlab # [ 7.705028] systemd[1]: Starting Rebuild Journal Catalog...770gitlab # [ 7.711879] systemd[1]: Starting Record System Boot/Shutdown in UTMP...771gitlab # [ 7.713365] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.772gitlab # [ 7.730015] systemd[1]: Finished Save Transient machine-id to Disk.773gitlab # [ 7.747407] systemd[1]: Finished Record System Boot/Shutdown in UTMP.774gitlab # [ 7.761995] systemd[1]: Finished Rebuild Journal Catalog.775gitlab # [ 7.764849] systemd[1]: Starting Update is Completed...776gitlab # [ 7.793756] systemd[1]: Finished Update is Completed.777gitlab # [ 7.800095] mousedev: PS/2 mouse device common for all mice778gitlab # [ 7.835922] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.779gitlab # [ 7.837454] systemd[1]: Finished Create SUID/SGID Wrappers.780gitlab # [ 7.838697] systemd[1]: suid-sgid-wrappers.service: Consumed 142ms CPU time over 213ms wall clock time, 6.3M memory peak, 1.9M read from disk.781gitlab # [ 8.105631] systemd[1]: Finished register-nix-paths.service.782gitlab # [ 8.107250] systemd[1]: Reached target System Initialization.783gitlab # [ 8.109764] systemd[1]: Started Discard unused filesystem blocks once a week.784gitlab # [ 8.111663] systemd[1]: Started logrotate.timer.785gitlab # [ 8.113547] systemd[1]: Started Daily Cleanup of Temporary Directories.786gitlab # [ 8.115189] systemd[1]: Reached target Timer Units.787gitlab # [ 8.115973] systemd[1]: Listening on D-Bus System Message Bus Socket.788gitlab # [ 8.116992] systemd[1]: Listening on Nix Daemon Socket.789gitlab # [ 8.121237] systemd[1]: Starting nixbot CI service socket...790gitlab # [ 8.122548] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.791gitlab # [ 8.123965] systemd[1]: Listening on nixbot CI service socket.792gitlab # [ 8.125204] systemd[1]: Reached target Socket Units.793gitlab # [ 8.126276] systemd[1]: Starting D-Bus System Message Bus...794gitlab # [ 8.160400] dbus-broker-launch[566]: Looking up NSS user entry for 'systemd-timesync'...795gitlab # [ 8.162337] dbus-broker-launch[566]: NSS returned no entry for 'systemd-timesync'796gitlab # [ 8.163825] dbus-broker-launch[566]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"797gitlab # [ 8.168558] systemd[1]: Started D-Bus System Message Bus.798gitlab # [ 8.179429] dbus-broker-launch[566]: Ready799gitlab # [ 8.180857] systemd[1]: Reached target Basic System.800gitlab # [ 8.182873] systemd[1]: Started backdoor.service.801gitlab # [ 8.188405] systemd[1]: Starting gitlab-config.service...802gitlab # [ 8.200187] systemd[1]: Starting Import lastlog data into lastlog2 database...803gitlab # [ 8.206787] systemd[1]: Starting Logrotate configuration check...804gitlab # [ 8.216435] systemd[1]: Starting Name Service Cache Daemon (nsncd)...805gitlab # [ 8.221616] systemd[1]: Starting Post-Boot Actions...806gitlab # [ 8.224309] logrotate[571]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.807gitlab # [ 8.232429] logrotate[571]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf808gitlab # [ 8.233842] logrotate[571]: note: 'monthly' overrides previously specified 'weekly'809gitlab # [ 8.234953] logrotate[571]: Reading state from file: /var/lib/logrotate.status810gitlab # connecting to host...811gitlab # [ 8.236722] logrotate[571]: state file /var/lib/logrotate.status does not exist812gitlab # [ 8.238243] logrotate[571]: Allocating hash table for state file, size 64 entries813gitlab # [ 8.239711] logrotate[571]: Handling 4 logs814gitlab # [ 8.240429] logrotate[571]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed815gitlab # [ 8.242848] logrotate[571]: considering log /var/log/btmp816gitlab # [ 8.243752] logrotate[571]: Creating new state817gitlab # [ 8.245778] logrotate[571]: Now: 2026-09-08 18:36818gitlab # [ 8.246496] logrotate[571]: Last rotated at 2026-09-08 18:00819gitlab # [ 8.248502] logrotate[571]: log does not need rotating (log has already been rotated)820gitlab # [ 8.250417] logrotate[571]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed821gitlab # [ 8.253480] logrotate[571]: considering log /var/log/wtmp822gitlab # [ 8.254224] logrotate[571]: Creating new state823gitlab # [ 8.254842] logrotate[571]: Now: 2026-09-08 18:36824gitlab # [ 8.256083] logrotate[571]: Last rotated at 2026-09-08 18:00825gitlab: Guest shell says: b'Spawning backdoor root shell...\n'826gitlab # [ 8.256898] logrotate[571]: log does not need rotating (log has already been rotated)827gitlab # [ 8.258060] logrotate[571]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed828gitlab: connected to guest root shell829gitlab # [ 8.259730] logrotate[571]: switching euid from 0 to 165 and egid from 0 to 165 (pid 571)830gitlab: (connecting took 8.73 seconds)831gitlab # [ 8.261745] logrotate[571]: considering log /var/gitlab/state/log/*.log832gitlab: (finished: waiting for the VM to finish booting, in 8.73 seconds)833gitlab # [ 8.262617] logrotate[571]: log /var/gitlab/state/log/*.log does not exist -- skipping834gitlab # [ 8.265977] logrotate[571]: Creating new state835gitlab # [ 8.266685] logrotate[571]: switching euid from 165 to 0 and egid from 165 to 0 (pid 571)836gitlab # [ 8.267893] logrotate[571]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed[ 8.272333] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3837gitlab # 838gitlab # [ 8.271953] logrotate[571]: switching euid from 0 to 60 and egid from 0 to 60 (pid 571)839gitlab # [ 8.276337] logrotate[571]: considering log /var/log/nginx/*.log840gitlab # [ 8.277215] logrotate[571]: log /var/log/nginx/*.log does not exist -- skipping841gitlab # [ 8.278315] logrotate[571]: Creating new state842gitlab # [ 8.278941] logrotate[571]: not running postrotate script, since no logs were rotated843gitlab # [ 8.280526] logrotate[571]: switching euid from 60 to 0 and egid from 60 to 0 (pid 571)844gitlab # [ 8.282080] nsncd[572]: Sep 08 18:36:09.021 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"845gitlab # [ 8.286616] systemd[1]: Started Reset console on configuration changes.846gitlab # [ 8.287850] systemd[1]: Starting resolvconf update...847gitlab # [ 8.288850] systemd[1]: Started Name Service Cache Daemon (nsncd).848gitlab # [ 8.289829] systemd[1]: Finished Logrotate configuration check.849gitlab # [ 8.290638] systemd[1]: Finished Post-Boot Actions.850gitlab # [ 8.291325] systemd[1]: Reached target Host and Network Name Lookups.851gitlab # [ 8.311316] systemd[1]: Reached target User and Group Name Lookups.852gitlab # [ 8.312366] systemd[1]: Starting User Login Management...853gitlab # [ 8.313304] systemd[1]: Finished Import lastlog data into lastlog2 database.854gitlab # [ 8.362722] systemd-logind[607]: New seat seat0.855gitlab # [ 8.391365] systemd-logind[607]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)856gitlab # [ 8.392841] systemd-logind[607]: Watching system buttons on /dev/input/event0 (gpio-keys)857gitlab # [ 8.395367] systemd[1]: Started User Login Management.858gitlab # [ 8.400815] systemd[1]: Starting linger-users.service...859gitlab # [ 8.407338] systemd[1]: Stopped target Host and Network Name Lookups.860gitlab # [ 8.408687] systemd[1]: Stopping Host and Network Name Lookups...861gitlab # [ 8.410612] systemd[1]: Stopped target User and Group Name Lookups.862gitlab # [ 8.411661] systemd[1]: Stopping User and Group Name Lookups...863gitlab # [ 8.413429] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...864gitlab # [ 8.414319] systemd[1]: nscd.service: Deactivated successfully.865gitlab # [ 8.416445] systemd[1]: Stopped Name Service Cache Daemon (nsncd).866gitlab # [ 8.417419] systemd[1]: nscd.service: Consumed 30ms CPU time over 195ms wall clock time, 7.5M memory peak, 5.8M read from disk.867gitlab # [ 8.426615] systemd[1]: Starting Name Service Cache Daemon (nsncd)...868gitlab # [ 8.433199] systemd[1]: linger-users.service: Deactivated successfully.869gitlab # [ 8.434663] systemd[1]: Finished linger-users.service.870gitlab # [ 8.448476] systemd[1]: Started Name Service Cache Daemon (nsncd).871gitlab # [ 8.449576] systemd[1]: Reached target Host and Network Name Lookups.872gitlab # [ 8.450894] nsncd[663]: Sep 08 18:36:09.219 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"873gitlab # [ 8.456593] systemd[1]: Reached target User and Group Name Lookups.874gitlab # [ 8.485208] systemd[1]: Finished resolvconf update.875gitlab # [ 8.487695] systemd[1]: Reached target Preparation for Network.876gitlab # [ 8.492657] systemd[1]: Starting DHCP Client...877gitlab # [ 8.497656] systemd[1]: Starting Address configuration of eth1...878gitlab # [ 8.500498] systemd[1]: Starting Extra networking commands....879gitlab # [ 8.518856] systemd-logind[607]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)880gitlab # [ 8.522867] systemd[1]: Starting Virtual Console Setup...881gitlab # [ 8.548874] network-addresses-eth1-start[694]: adding address 192.168.1.1/24... done882gitlab # [ 8.554884] network-addresses-eth1-start[694]: adding address 2001:db8:1::1/64... done883gitlab # [ 8.563224] systemd[1]: Finished Address configuration of eth1.884gitlab # [ 8.598454] systemd-vconsole-setup[696]: Configuration of first virtual console was skipped, ignoring remaining ones.885gitlab # [ 8.606013] systemd[1]: Finished Virtual Console Setup.886gitlab # [ 8.624880] dhcpcd[713]: dhcpcd-10.3.2 starting887gitlab # [ 8.625657] systemd[1]: Finished Extra networking commands..888gitlab # [ 8.626641] systemd[1]: Reached target Network.889gitlab # [ 8.627957] dhcpcd[762]: dev: loaded udev890gitlab # [ 8.632781] systemd[1]: Starting gitlab-workhorse.service...891gitlab # [ 8.636682] systemd[1]: Starting Nginx Web Server...892gitlab # [ 8.646256] 8021q: 802.1Q VLAN Support v1.8893gitlab # [ 8.645617] systemd[1]: Starting PostgreSQL Server...894gitlab # [ 8.646713] 8021q: adding VLAN 0 to HW filter on device eth1895gitlab # [ 8.649916] systemd[1]: Starting Redis Server - redis-gitlab...896gitlab # [ 8.654061] systemd[1]: Starting Permit User Sessions...897gitlab # [ 8.665680] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.898gitlab # [ 8.688217] systemd[1]: Finished Permit User Sessions.899gitlab # [ 8.698237] cfg80211: Loading compiled-in X.509 certificates for regulatory database900gitlab # [ 8.698439] systemd[1]: Started Getty on tty1.901gitlab # [ 8.699259] systemd[1]: Reached target Login Prompts.902gitlab # [ 8.713390] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'903gitlab # [ 8.713926] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'904gitlab # [ 8.716464] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2905gitlab # [ 8.716836] cfg80211: failed to load regulatory.db906gitlab # [ 8.765485] 8021q: adding VLAN 0 to HW filter on device eth0907gitlab # [ 8.763793] dhcpcd[762]: eth0: waiting for carrier908gitlab # [ 8.765419] dhcpcd[762]: eth0: carrier acquired909gitlab # [ 8.768465] dhcpcd[762]: DUID 00:01:00:01:32:33:12:19:52:54:00:12:34:56910gitlab # [ 8.769646] dhcpcd[762]: eth0: IAID 00:12:34:56911gitlab # [ 8.770337] dhcpcd[762]: eth0: adding address fe80::5054:ff:fe12:3456912gitlab # [ 8.817961] nginx-pre-start[795]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok913gitlab # [ 8.820580] nginx-pre-start[795]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful914gitlab # [ 8.825343] systemd[1]: Started Nginx Web Server.915gitlab # [ 8.844779] redis-server[791]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.916gitlab # [ 8.847756] redis-server[791]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo917gitlab # [ 8.849183] redis-server[791]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=791, just started918gitlab # [ 8.850844] redis-server[791]: Configuration loaded919gitlab # [ 8.851745] systemd[1]: Started Redis Server - redis-gitlab.920gitlab # [ 8.852664] redis[791]: monotonic clock: ARM CNTVCT @ 1000 ticks/us921gitlab # [ 8.853731] redis[791]: Running mode=standalone, port=0.922gitlab # [ 8.854921] redis[791]: Server initialized923gitlab # [ 8.855757] redis[791]: Ready to accept connections unix924gitlab # [ 8.857030] redis[791]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.925gitlab # [ 8.865828] postgresql-pre-start[796]: The files belonging to this database system will be owned by user "postgres".926gitlab # [ 8.867536] postgresql-pre-start[796]: This user must also own the server process.927gitlab # [ 8.869995] postgresql-pre-start[796]: The database cluster will be initialized with locale "en_US.UTF-8".928gitlab # [ 8.871986] postgresql-pre-start[796]: The default database encoding has accordingly been set to "UTF8".929gitlab # [ 8.874643] postgresql-pre-start[796]: The default text search configuration will be set to "english".930gitlab # [ 8.876980] postgresql-pre-start[796]: Data page checksums are enabled.931gitlab # [ 8.880242] postgresql-pre-start[796]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok932gitlab # [ 8.881914] postgresql-pre-start[796]: creating subdirectories ... ok933gitlab # [ 8.882987] postgresql-pre-start[796]: selecting dynamic shared memory implementation ... posix934gitlab # [ 8.915736] postgresql-pre-start[796]: selecting default "max_connections" ... 100935gitlab # [ 8.976114] postgresql-pre-start[796]: selecting default "shared_buffers" ... 128MB936gitlab # [ 9.291631] systemd[1]: Started gitlab-workhorse.service.937gitlab # [ 9.506192] postgresql-pre-start[796]: selecting default time zone ... UTC938gitlab # [ 9.510009] postgresql-pre-start[796]: creating configuration files ... ok939gitlab # [ 9.737137] gitlab-workhorse[823]: time="2026-09-08T18:36:10Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4940gitlab # [ 9.738801] gitlab-workhorse[823]: time="2026-09-08T18:36:10Z" level=info msg="Using redis/go-redis"941gitlab # [ 9.740122] gitlab-workhorse[823]: time="2026-09-08T18:36:10Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix942gitlab # [ 9.811268] postgresql-pre-start[796]: running bootstrap script ... ok943gitlab # [ 10.440254] postgresql-pre-start[796]: performing post-bootstrap initialization ... ok944gitlab # [ 10.454682] dhcpcd[762]: eth0: soliciting a DHCP lease945gitlab # [ 10.464845] dhcpcd[762]: eth0: offered 10.0.2.15 from 10.0.2.2946gitlab # [ 10.484561] dhcpcd[762]: eth0: probing address 10.0.2.15/24947gitlab # [ 10.773623] postgresql-pre-start[796]: syncing data to disk ... ok948gitlab # [ 10.776601] postgresql-pre-start[796]: initdb: warning: enabling "trust" authentication for local connections949gitlab # [ 10.781013] postgresql-pre-start[796]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.950gitlab # [ 10.787683] postgresql-pre-start[796]: Success. You can now start the database server using:951gitlab # [ 10.791423] postgresql-pre-start[796]: pg_ctl -D /var/lib/postgresql/18 -l logfile start952gitlab # [ 10.971162] postgres[857]: [857] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit953gitlab # [ 10.976828] postgres[857]: [857] LOG: listening on IPv6 address "::1", port 5432954gitlab # [ 10.980426] postgres[857]: [857] LOG: listening on IPv4 address "127.0.0.1", port 5432955gitlab # [ 10.985235] postgres[857]: [857] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"956gitlab # [ 10.991344] postgres[866]: [866] LOG: database system was shut down at 2026-09-08 18:36:11 GMT957gitlab # [ 11.002568] postgres[857]: [857] LOG: database system is ready to accept connections958gitlab # [ 11.007481] systemd[1]: Started PostgreSQL Server.959gitlab # [ 11.024835] systemd[1]: Starting PostgreSQL Setup Scripts...960gitlab # [ 11.359393] postgresql-setup-start[881]: CREATE DATABASE961gitlab # [ 11.371613] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[877]: mkdir -p /var/gitlab/state/home/.ssh: OK962gitlab # [ 11.375817] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[877]: chmod 700 /var/gitlab/state/home/.ssh: OK963gitlab # [ 11.414089] postgresql-setup-start[891]: CREATE ROLE964gitlab # [ 11.433257] postgresql-setup-start[898]: ALTER DATABASE965gitlab # [ 11.453646] systemd[1]: Finished gitlab-config.service.966gitlab # [ 11.459972] systemd[1]: Started gitaly.service.967gitlab # [ 11.483348] postgresql-setup-start[907]: CREATE ROLE968gitlab # [ 11.493112] systemd[1]: Finished PostgreSQL Setup Scripts.969gitlab # [ 11.495463] systemd[1]: Reached target PostgreSQL.970gitlab # [ 11.500780] systemd[1]: Starting gitlab-db-config.service...971gitlab # [ 11.506656] systemd[1]: Starting gitlab-postgresql.service...972gitlab # [ 11.666423] gitlab-postgresql-start[927]: CREATE DATABASE973gitlab # [ 11.713135] dhcpcd[762]: eth0: soliciting an IPv6 router974gitlab # [ 11.717016] dhcpcd[762]: eth0: Router Advertisement from fe80::2975gitlab # [ 11.721042] dhcpcd[762]: eth0: adding address fec0::5054:ff:fe12:3456/64976gitlab # [ 11.722288] dhcpcd[762]: eth0: adding route to fec0::/64977gitlab # [ 11.723294] dhcpcd[762]: eth0: adding default route via fe80::2978gitlab # [ 11.726109] gitlab-postgresql-start[937]: CREATE EXTENSION979gitlab # [ 11.817148] gitlab-postgresql-start[941]: CREATE EXTENSION980gitlab # [ 11.821512] systemd[1]: Finished gitlab-postgresql.service.981gitlab # [ 12.174789] gitaly[906]: time="2026-09-08T18:36:12.944Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=906982gitlab # [ 12.192397] gitaly[906]: time="2026-09-08T18:36:12.944Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=906983gitlab # [ 12.195829] gitaly[906]: time="2026-09-08T18:36:12.944Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=906984gitlab # [ 12.204264] gitaly[906]: time="2026-09-08T18:36:12.944Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=906985gitlab # [ 12.208165] gitaly[906]: time="2026-09-08T18:36:12.944Z" level=info msg="Starting Gitaly" pid=906 version=986gitlab # [ 12.428996] gitaly[906]: time="2026-09-08T18:36:13.198Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=253 pid=906987gitlab # [ 12.431018] gitaly[906]: time="2026-09-08T18:36:13.199Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=906988gitlab # [ 12.432956] gitaly[906]: time="2026-09-08T18:36:13.200Z" level=info msg="finished initializing command factory" duration_ms=0 pid=906989gitlab # [ 12.434588] gitaly[906]: time="2026-09-08T18:36:13.200Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-906/git-exec-3317781065.d/git pid=906990gitlab # [ 12.441465] gitaly[906]: time="2026-09-08T18:36:13.211Z" level=info msg="finished detecting git version" duration_ms=11 pid=906991gitlab # [ 12.443488] gitaly[906]: time="2026-09-08T18:36:13.211Z" level=info msg="using Git version" pid=906 version=2.53.g992gitlab # [ 12.445431] gitaly[906]: time="2026-09-08T18:36:13.211Z" level=info msg="clearing disk cache object folder" pid=906 storage=default993gitlab # [ 12.447462] gitaly[906]: time="2026-09-08T18:36:13.212Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache969026371 pid=906 storage=default994gitlab # [ 12.450858] gitaly[906]: time="2026-09-08T18:36:13.212Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache969026371 pid=906 storage=default995gitlab # [ 12.454655] gitaly[906]: time="2026-09-08T18:36:13.212Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=906996gitlab # [ 12.458364] gitaly[906]: time="2026-09-08T18:36:13.212Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=906997gitlab # [ 12.461539] gitaly[906]: time="2026-09-08T18:36:13.213Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache969026371 clear_duration_ms=0 pid=906 storage=default998gitlab # [ 12.465103] gitaly[906]: time="2026-09-08T18:36:13.228Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=906999gitlab # [ 12.467754] gitaly[906]: time="2026-09-08T18:36:13.228Z" level=info msg="finished tempdir cleaner walk" pid=906 storage=default time_ms=01000gitlab # [ 12.470236] gitaly[906]: time="2026-09-08T18:36:13.228Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=906 schema=unix1001gitlab # [ 12.472876] gitaly[906]: time="2026-09-08T18:36:13.228Z" level=info msg="listening at address" address=/run/gitaly/gitaly-906/sock.d/intern pid=906 schema=unix1002gitlab # [ 12.475483] gitaly[906]: time="2026-09-08T18:36:13.228Z" level=info msg="counting repositories" pid=9061003gitlab # [ 12.477315] gitaly[906]: time="2026-09-08T18:36:13.229Z" level=info msg="starting to count repositories" pid=906 storage_path=/var/gitlab/state/repositories1004gitlab # [ 12.479906] gitaly[906]: time="2026-09-08T18:36:13.229Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=906 storage_path=/var/gitlab/state/repositories1005gitlab # [ 12.483068] gitaly[906]: time="2026-09-08T18:36:13.229Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9061006gitlab # [ 12.485486] gitaly[906]: time="2026-09-08T18:36:13.231Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9061007gitlab # [ 12.487833] gitaly[906]: time="2026-09-08T18:36:13.231Z" level=info msg="Started Gitaly" duration_ms=286 pid=9061008gitlab # [ 12.489905] gitaly[906]: time="2026-09-08T18:36:13.231Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9061009gitlab # [ 12.492346] gitaly[906]: time="2026-09-08T18:36:13.231Z" level=info msg="maintenance: daily scheduled" pid=906 scheduled="2026-09-09 12:00:00 +0000 UTC"1010gitlab # [ 12.808743] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1011gitlab # [ 12.818976] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1012gitlab # [ 12.828749] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1013gitlab # [ 15.123785] dhcpcd[762]: eth0: leased 10.0.2.15 for 86400 seconds1014gitlab # [ 15.126744] dhcpcd[762]: eth0: adding route to 10.0.2.0/241015gitlab # [ 15.129167] dhcpcd[762]: eth0: adding default route via 10.0.2.21016gitlab # [ 15.357990] systemd[1]: Started DHCP Client.1017gitlab # [ 15.368577] systemd[1]: Reached target Network is Online.1018gitlab # [ 16.140594] gitaly[906]: time="2026-09-08T18:36:16.907Z" level=info msg="License database preloaded" duration_ms=3962 pid=9061019gitlab # [ 58.377696] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[925]: W, [2026-09-08T18:36:59.146380 #925] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1020gitlab # [ 59.841342] gitlab-workhorse[823]: time="2026-09-08T18:37:00Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1021gitlab # [ 61.558255] postgres[1440]: [1440] ERROR: relation "feature_gates" does not exist at character 5231022gitlab # [ 61.559600] postgres[1440]: [1440] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1023gitlab # [ 61.560918] postgres[1440]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1024gitlab # [ 61.562204] postgres[1440]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1025gitlab # [ 61.563346] postgres[1440]: attidentity AS identity,1026gitlab # [ 61.564147] postgres[1440]: attgenerated as attgenerated1027gitlab # [ 61.564961] postgres[1440]: FROM pg_attribute a1028gitlab # [ 61.565630] postgres[1440]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1029gitlab # [ 61.566830] postgres[1440]: LEFT JOIN pg_type t ON a.atttypid = t.oid1030gitlab # [ 61.567724] postgres[1440]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1031gitlab # [ 61.569101] postgres[1440]: WHERE a.attrelid = '"feature_gates"'::regclass1032gitlab # [ 61.570039] postgres[1440]: AND a.attnum > 0 AND NOT a.attisdropped1033gitlab # [ 61.570926] postgres[1440]: ORDER BY a.attnum1034gitlab # [ 61.571568] postgres[1440]:1035gitlab # [ 66.766666] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1491]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1036gitlab # [ 66.767093] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1491]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1037gitlab # [ 69.851017] gitlab-workhorse[823]: time="2026-09-08T18:37:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 79.858023] gitlab-workhorse[823]: time="2026-09-08T18:37:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1039gitlab # [ 89.868959] gitlab-workhorse[823]: time="2026-09-08T18:37:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1040gitlab # [ 93.968147] postgres[1594]: [1594] ERROR: relation "table_without_models" does not exist1041gitlab # [ 93.972307] postgres[1594]: [1594] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1042gitlab # [ 99.879121] gitlab-workhorse[823]: time="2026-09-08T18:37:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1043gitlab # [ 109.886228] gitlab-workhorse[823]: time="2026-09-08T18:37:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1044gitlab # [ 119.894480] gitlab-workhorse[823]: time="2026-09-08T18:38:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1045gitlab # [ 124.323136] systemd[1]: Finished gitlab-db-config.service.1046gitlab # [ 124.328808] systemd[1]: Started gitlab-sidekiq.service.1047gitlab # [ 124.335610] systemd[1]: Starting gitlab.service...1048gitlab # [ 125.012609] bundle[1987]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1049gitlab # [ 125.013131] bundle[1987]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1050gitlab # [ 125.016108] bundle[1987]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1051gitlab # [ 129.731767] sidekiq-cluster[1986]: {"severity":"INFO","time":"2026-09-08T18:38:10.473Z","message":"Starting cluster with 1 processes"}1052gitlab # [ 129.812434] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.552Z","pid":1987,"message":"* Enabling systemd notification integration"}1053gitlab # [ 129.817316] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"Puma starting in cluster mode..."}1054gitlab # [ 129.821667] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1055gitlab # [ 129.826354] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1056gitlab # [ 129.832298] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Min threads: 0"}1057gitlab # [ 129.836112] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Max threads: 4"}1058gitlab # [ 129.839977] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Environment: production"}1059gitlab # [ 129.844205] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Master PID: 1987"}1060gitlab # [ 129.848245] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Workers: 2"}1061gitlab # [ 129.852443] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1062gitlab # [ 129.857698] bundle[1987]: {"timestamp":"2026-09-08T18:38:10.581Z","pid":1987,"message":"* Preloading application"}1063gitlab # [ 129.902221] gitlab-workhorse[823]: time="2026-09-08T18:38:10Z" level=error correlation_id= duration_ms=5 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1064gitlab # [ 139.903528] gitlab-workhorse[823]: time="2026-09-08T18:38:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1065gitlab # [ 149.904805] gitlab-workhorse[823]: time="2026-09-08T18:38:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1066gitlab # [ 159.906088] gitlab-workhorse[823]: time="2026-09-08T18:38:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1067gitlab # [ 167.675756] bundle[1987]: W, [2026-09-08T18:38:48.444634 #1987] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1068gitlab # [ 169.907120] gitlab-workhorse[823]: time="2026-09-08T18:38:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1069gitlab # [ 171.452092] sidekiq-cluster[2035]: 2026-09-08T18:38:52.220Z pid=2035 tid=wnv INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2035, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433373184}1070gitlab # [ 171.747823] sidekiq-cluster[2035]: W, [2026-09-08T18:38:52.517841 #2035] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1071gitlab # [ 171.814115] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:52.584Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1072gitlab # [ 175.636551] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.405Z","message":"GitLab reliable fetch activated!"}1073gitlab # [ 175.641544] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.405Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1074gitlab # [ 175.669650] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.439Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1075gitlab # [ 175.676620] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.442Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1076gitlab # [ 175.683597] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.445Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1077gitlab # [ 175.690282] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.448Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1078gitlab # [ 175.697683] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.450Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1079gitlab # [ 175.704278] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.453Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1080gitlab # [ 175.710435] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.456Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1081gitlab # [ 175.715609] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.458Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1082gitlab # [ 175.720388] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.460Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1083gitlab # [ 175.724739] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.462Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1084gitlab # [ 175.729321] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.464Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1085gitlab # [ 175.733416] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.467Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1086gitlab # [ 175.737006] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.469Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1087gitlab # [ 175.740665] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.471Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1088gitlab # [ 175.744101] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.473Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1089gitlab # [ 175.747390] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.475Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1090gitlab # [ 175.750514] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.478Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1091gitlab # [ 175.753388] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.481Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1092gitlab # [ 175.756422] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.484Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1093gitlab # [ 175.759237] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.487Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1094gitlab # [ 175.762360] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.489Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1095gitlab # [ 175.765390] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.492Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1096gitlab # [ 175.767914] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.494Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1097gitlab # [ 175.770605] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.496Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1098gitlab # [ 175.773263] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.498Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1099gitlab # [ 175.775649] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.501Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1100gitlab # [ 175.778031] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.504Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1101gitlab # [ 175.780329] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.507Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1102gitlab # [ 175.782617] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.510Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1103gitlab # [ 175.784976] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.513Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1104gitlab # [ 175.787088] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.515Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1105gitlab # [ 175.789560] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.518Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1106gitlab # [ 175.791684] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.521Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1107gitlab # [ 175.794005] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.524Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1108gitlab # [ 175.796124] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.526Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1109gitlab # [ 175.798202] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.529Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1110gitlab # [ 175.800343] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.531Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1111gitlab # [ 175.802337] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.533Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1112gitlab # [ 175.804388] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.535Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1113gitlab # [ 175.806388] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.537Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1114gitlab # [ 175.808576] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.539Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1115gitlab # [ 175.810657] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.541Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1116gitlab # [ 175.812847] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.543Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1117gitlab # [ 175.814868] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.545Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1118gitlab # [ 175.817079] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.548Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1119gitlab # [ 175.819322] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.550Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1120gitlab # [ 175.821516] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.552Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1121gitlab # [ 175.823556] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.555Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1122gitlab # [ 175.825667] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.557Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1123gitlab # [ 175.827690] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.559Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1124gitlab # [ 175.829820] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.561Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1125gitlab # [ 175.831825] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.564Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1126gitlab # [ 175.834025] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.576Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1127gitlab # [ 175.836378] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.579Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1128gitlab # [ 175.838513] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.582Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1129gitlab # [ 175.840692] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.584Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1130gitlab # [ 175.842814] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.586Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1131gitlab # [ 175.845152] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.589Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1132gitlab # [ 175.847330] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.591Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1133gitlab # [ 175.849638] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.593Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1134gitlab # [ 175.851618] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.595Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1135gitlab # [ 175.854719] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.597Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1136gitlab # [ 175.856886] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.599Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1137gitlab # [ 175.859022] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.602Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1138gitlab # [ 175.861308] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.605Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1139gitlab # [ 175.863444] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.607Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1140gitlab # [ 175.865578] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.609Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1141gitlab # [ 175.867694] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.611Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1142gitlab # [ 175.870060] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.613Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1143gitlab # [ 175.872369] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.614Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1144gitlab # [ 175.874516] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.616Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1145gitlab # [ 175.876694] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.618Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1146gitlab # [ 175.878813] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.620Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1147gitlab # [ 175.881114] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.622Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1148gitlab # [ 175.883378] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.624Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1149gitlab # [ 175.885792] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.626Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1150gitlab # [ 175.887979] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.628Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1151gitlab # [ 175.890580] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.630Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1152gitlab # [ 175.892787] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.632Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1153gitlab # [ 175.894872] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.634Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1154gitlab # [ 175.897252] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.635Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1155gitlab # [ 175.899691] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.637Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1156gitlab # [ 175.902083] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.640Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1157gitlab # [ 175.904305] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.642Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1158gitlab # [ 175.906602] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.644Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1159gitlab # [ 175.908834] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.646Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1160gitlab # [ 175.911117] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.648Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1161gitlab # [ 175.913403] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.650Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1162gitlab # [ 175.915643] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.652Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1163gitlab # [ 175.917984] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.654Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1164gitlab # [ 175.920231] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.656Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1165gitlab # [ 175.922429] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.658Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1166gitlab # [ 175.924661] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.659Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1167gitlab # [ 175.926951] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.661Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1168gitlab # [ 175.928909] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.664Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1169gitlab # [ 175.931123] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.667Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1170gitlab # [ 175.933398] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.669Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1171gitlab # [ 175.935595] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.672Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1172gitlab # [ 175.937907] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.684Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1173gitlab # [ 175.940118] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.687Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1174gitlab # [ 175.942244] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.689Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1175gitlab # [ 175.944495] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.692Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1176gitlab # [ 175.946522] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.695Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1177gitlab # [ 175.948727] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.698Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1178gitlab # [ 175.951022] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.701Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1179gitlab # [ 175.953406] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.703Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1180gitlab # [ 175.977259] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.747Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1181gitlab # [ 175.980807] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.750Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1182gitlab # [ 175.983021] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:38:56.752Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1183gitlab # [ 179.908255] gitlab-workhorse[823]: time="2026-09-08T18:39:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1184gitlab # [ 189.919116] gitlab-workhorse[823]: time="2026-09-08T18:39:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1185gitlab # [ 199.926207] gitlab-workhorse[823]: time="2026-09-08T18:39:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1186gitlab # [ 209.933707] gitlab-workhorse[823]: time="2026-09-08T18:39:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1187gitlab # [ 216.441017] postgres[2765]: [2765] ERROR: relation "table_without_models" does not exist1188gitlab # [ 216.441389] postgres[2765]: [2765] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1189gitlab # [ 218.274789] postgres[2785]: [2785] ERROR: relation "table_without_models" does not exist1190gitlab # [ 218.275114] postgres[2785]: [2785] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1191gitlab # [ 219.942138] gitlab-workhorse[823]: time="2026-09-08T18:39:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1192gitlab # [ 221.435313] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:42.204Z","message":"Booted Rails 7.2.3.1 application in production environment"}1193gitlab # [ 221.440188] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:42.208Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1194gitlab # [ 221.446437] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:42.208Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1195gitlab # [ 221.451919] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:42.208Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1196gitlab # [ 221.458131] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:42.215Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1197gitlab # [ 223.906393] bundle[1987]: {"timestamp":"2026-09-08T18:39:44.675Z","pid":1987,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1198gitlab # [ 223.912256] bundle[1987]: {"timestamp":"2026-09-08T18:39:44.675Z","pid":1987,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1199gitlab # [ 223.912402] bundle[1987]: {"timestamp":"2026-09-08T18:39:44.675Z","pid":1987,"message":"! #\u003cThread:0x0000ecfc8b36d718 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1200gitlab # [ 223.912537] bundle[1987]: {"timestamp":"2026-09-08T18:39:44.675Z","pid":1987,"message":"! #\u003cThread:0x0000ecfc8b543f38 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1201gitlab # [ 223.912654] bundle[1987]: {"timestamp":"2026-09-08T18:39:44.675Z","pid":1987,"message":"Use Ctrl-C to stop"}1202gitlab # [ 223.912750] bundle[1987]: {"timestamp":"2026-09-08T18:39:44.676Z","pid":1987,"message":"Sending status to systemd every 1.0 sec"}1203gitlab # [ 224.193994] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:44.963Z","message":"Cleaning working queues"}1204gitlab # [ 224.212466] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:44.982Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"44b424602a24b758bbd3e600","created_at":"2026-09-08T18:38:04.471Z","trace_propagation_headers":{"sentry-trace":"1d860cecaa1e4add99cb4d5762c76393-7fb0157b65a94760","baggage":"sentry-trace_id=1d860cecaa1e4add99cb4d5762c76393,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b2df1acb8c15753bd5ab2e906810edc8","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T18:38:04.482Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-44b424602a24b758bbd3e600: start","job_status":"start","queue_duration_s":100.499955,"scheduling_latency_s":100.500012}1205gitlab # [ 224.535974] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:45.305Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"44b424602a24b758bbd3e600","created_at":"2026-09-08T18:38:04.471Z","trace_propagation_headers":{"sentry-trace":"1d860cecaa1e4add99cb4d5762c76393-7fb0157b65a94760","baggage":"sentry-trace_id=1d860cecaa1e4add99cb4d5762c76393,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b2df1acb8c15753bd5ab2e906810edc8","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T18:38:04.482Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-44b424602a24b758bbd3e600: done: 0.323052 sec","job_status":"done","queue_duration_s":100.499955,"scheduling_latency_s":100.500012,"redis_calls":24,"redis_duration_s":0.00915,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.004299,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00309,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.174733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1763365,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.323052,"completed_at":"2026-09-08T18:39:45.304Z","load_balancing_strategy":"primary","db_duration_s":0.076333,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1206gitlab # [ 224.601947] bundle[1987]: {"timestamp":"2026-09-08T18:39:45.322Z","pid":1987,"message":"- Worker 0 (PID: 2877) booted in 0.6s, phase: 0"}1207gitlab # [ 224.603704] bundle[1987]: {"timestamp":"2026-09-08T18:39:45.349Z","pid":1987,"message":"- Worker 1 (PID: 2881) booted in 0.6s, phase: 0"}1208gitlab # [ 224.605867] systemd[1]: Started gitlab.service.1209gitlab # [ 224.606674] systemd[1]: Reached target Common target for all GitLab services..1210gitlab # [ 224.607755] systemd[1]: Starting setup-gitlab.service...1211gitlab # [ 224.643317] setup-gitlab-start[2921]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1212gitlab # [ 229.157634] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:39:49 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 44631213gitlab # [ 229.165205] setup-gitlab-start[2921]: + TOKEN=nixbot-test-token-of-20-chars1214gitlab # [ 229.166265] setup-gitlab-start[2921]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1215gitlab # [ 229.183542] runuser[2948]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1216gitlab # [ 229.744774] setup-gitlab-start[2949]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1217gitlab # [ 229.745063] setup-gitlab-start[2949]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1218gitlab # [ 229.746678] setup-gitlab-start[2949]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1219gitlab # [ 230.563357] setup-gitlab-start[2949]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1220gitlab # [ 230.563724] setup-gitlab-start[2949]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1221gitlab # [ 230.566054] setup-gitlab-start[2949]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1222gitlab # [ 239.157055] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:39:59.925Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"56026577e025200fffcbcefc","created_at":"2026-09-08T18:39:59.813Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81979abd43ca0385e9ea68c228850e10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:39:59.921Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-56026577e025200fffcbcefc: start","job_status":"start","queue_duration_s":0.003223,"scheduling_latency_s":0.003282}1223gitlab # [ 239.350598] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.119Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"56026577e025200fffcbcefc","created_at":"2026-09-08T18:39:59.813Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81979abd43ca0385e9ea68c228850e10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:39:59.921Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-56026577e025200fffcbcefc: done: 0.19428 sec","job_status":"done","queue_duration_s":0.003223,"scheduling_latency_s":0.003282,"redis_calls":12,"redis_duration_s":0.02504,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006176,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00849482,"duration_s":0.19428,"completed_at":"2026-09-08T18:40:00.119Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046918,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1224gitlab # [ 239.442573] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.130Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"13f935cd5708c61f73246c41","created_at":"2026-09-08T18:39:59.945Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5123c1384663f141c408e5f763369f9c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:39:59.979Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-13f935cd5708c61f73246c41: start","job_status":"start","queue_duration_s":0.150772,"scheduling_latency_s":0.150824}1225gitlab # [ 239.573541] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"13f935cd5708c61f73246c41","created_at":"2026-09-08T18:39:59.945Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5123c1384663f141c408e5f763369f9c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:39:59.979Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-13f935cd5708c61f73246c41: done: 0.211978 sec","job_status":"done","queue_duration_s":0.150772,"scheduling_latency_s":0.150824,"redis_calls":16,"redis_duration_s":0.04528,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.014447,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.029188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001645,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.06,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00180216,"duration_s":0.211978,"completed_at":"2026-09-08T18:40:00.342Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1226gitlab # [ 239.665038] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8793e858e66b4917b7f9cf41","created_at":"2026-09-08T18:39:59.995Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00d7864286499c2ed7c2f2390199dd3b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.008Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8793e858e66b4917b7f9cf41: start","job_status":"start","queue_duration_s":0.361608,"scheduling_latency_s":0.361662}1227gitlab # [ 239.834115] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.602Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8793e858e66b4917b7f9cf41","created_at":"2026-09-08T18:39:59.995Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00d7864286499c2ed7c2f2390199dd3b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.008Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8793e858e66b4917b7f9cf41: done: 0.232522 sec","job_status":"done","queue_duration_s":0.361608,"scheduling_latency_s":0.361662,"redis_calls":13,"redis_duration_s":0.048748,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002091,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00908,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.037577,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03251378,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.232522,"completed_at":"2026-09-08T18:40:00.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012811,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1228gitlab # [ 239.938840] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"40d5c018cde1c300c2901451","created_at":"2026-09-08T18:40:00.041Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37f6d64d8e2cecd35a46ecafc2b3e1c5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.066Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-40d5c018cde1c300c2901451: start","job_status":"start","queue_duration_s":0.55677,"scheduling_latency_s":0.556826}1229gitlab # [ 240.027769] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"40d5c018cde1c300c2901451","created_at":"2026-09-08T18:40:00.041Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37f6d64d8e2cecd35a46ecafc2b3e1c5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.066Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-40d5c018cde1c300c2901451: done: 0.173036 sec","job_status":"done","queue_duration_s":0.55677,"scheduling_latency_s":0.556826,"redis_calls":10,"redis_duration_s":0.046591,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001947,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.04178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002864,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00303372,"duration_s":0.173036,"completed_at":"2026-09-08T18:40:00.796Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.061764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1230gitlab # [ 240.115817] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:00.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"eb3dafec83435a00c4208608","created_at":"2026-09-08T18:40:00.095Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.123Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-eb3dafec83435a00c4208608: start","job_status":"start","queue_duration_s":0.687378,"scheduling_latency_s":0.68744}1231gitlab # [ 240.264475] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"eb3dafec83435a00c4208608","created_at":"2026-09-08T18:40:00.095Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.123Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-eb3dafec83435a00c4208608: done: 0.22097 sec","job_status":"done","queue_duration_s":0.687378,"scheduling_latency_s":0.68744,"redis_calls":19,"redis_duration_s":0.075924,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003296,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000851,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.068418,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00067668,"duration_s":0.22097,"completed_at":"2026-09-08T18:40:01.031Z","load_balancing_strategy":"primary","db_duration_s":0.040422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1232gitlab # [ 240.352113] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"dc4d585491fe291f58cbfb2c","created_at":"2026-09-08T18:40:00.137Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de9d3309c39ce0007e1319b65744e07a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.186Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-dc4d585491fe291f58cbfb2c: start","job_status":"start","queue_duration_s":0.849168,"scheduling_latency_s":0.849221}1233gitlab # [ 240.369994] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"dc4d585491fe291f58cbfb2c","created_at":"2026-09-08T18:40:00.137Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de9d3309c39ce0007e1319b65744e07a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.186Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-dc4d585491fe291f58cbfb2c: done: 0.051054 sec","job_status":"done","queue_duration_s":0.849168,"scheduling_latency_s":0.849221,"redis_calls":12,"redis_duration_s":0.0041270000000000005,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000784,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001413,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00098504,"exclusive_lock_hold_duration_s":0.00658524,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.051054,"completed_at":"2026-09-08T18:40:01.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1234gitlab # [ 240.417797] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.089Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"126077a2e584697e844ec41b","created_at":"2026-09-08T18:40:00.221Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8087384fef1261707c8c2fcc77450ef6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T18:40:00.243Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-126077a2e584697e844ec41b: start","job_status":"start","queue_duration_s":0.845392,"scheduling_latency_s":0.845418}1235gitlab # [ 240.431694] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"126077a2e584697e844ec41b","created_at":"2026-09-08T18:40:00.221Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8087384fef1261707c8c2fcc77450ef6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T18:40:00.243Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-126077a2e584697e844ec41b: done: 0.047246 sec","job_status":"done","queue_duration_s":0.845392,"scheduling_latency_s":0.845418,"redis_calls":10,"redis_duration_s":0.002928,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043668,"duration_s":0.047246,"completed_at":"2026-09-08T18:40:01.136Z","load_balancing_strategy":"primary","db_duration_s":0.004799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1236gitlab # [ 240.471689] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e93ff16aa9b7ad95e13b13cb","created_at":"2026-09-08T18:40:00.258Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b2927661b675dbde5c067c8bcf4f2a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T18:40:00.280Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e93ff16aa9b7ad95e13b13cb: start","job_status":"start","queue_duration_s":0.857651,"scheduling_latency_s":0.857688}1237gitlab # [ 240.485003] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.151Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1238gitlab # [ 240.488187] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.160Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e93ff16aa9b7ad95e13b13cb","created_at":"2026-09-08T18:40:00.258Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b2927661b675dbde5c067c8bcf4f2a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T18:40:00.280Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e93ff16aa9b7ad95e13b13cb: done: 0.021985 sec","job_status":"done","queue_duration_s":0.857651,"scheduling_latency_s":0.857688,"redis_calls":10,"redis_duration_s":0.002565,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001602,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032886,"duration_s":0.021985,"completed_at":"2026-09-08T18:40:01.160Z","load_balancing_strategy":"primary","db_duration_s":0.000533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1239gitlab # [ 240.528200] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.162Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f04cdf9b289788d2df10a7b1","created_at":"2026-09-08T18:40:00.330Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3385400100e15536d6cb586b16816775","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.349Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f04cdf9b289788d2df10a7b1: start","job_status":"start","queue_duration_s":0.81246,"scheduling_latency_s":0.812476}1240gitlab # [ 240.542202] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.175Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1241gitlab # [ 240.545797] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f04cdf9b289788d2df10a7b1","created_at":"2026-09-08T18:40:00.330Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3385400100e15536d6cb586b16816775","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.349Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f04cdf9b289788d2df10a7b1: done: 0.021547 sec","job_status":"done","queue_duration_s":0.81246,"scheduling_latency_s":0.812476,"redis_calls":10,"redis_duration_s":0.002817,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000909,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003368,"duration_s":0.021547,"completed_at":"2026-09-08T18:40:01.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1242gitlab # [ 240.587962] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3a4cb0d93ee5105b0ee2f94b","created_at":"2026-09-08T18:40:00.361Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.376Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3a4cb0d93ee5105b0ee2f94b: start","job_status":"start","queue_duration_s":0.808908,"scheduling_latency_s":0.808925}1243gitlab # [ 240.601373] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3a4cb0d93ee5105b0ee2f94b","created_at":"2026-09-08T18:40:00.361Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.376Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3a4cb0d93ee5105b0ee2f94b: done: 0.122715 sec","job_status":"done","queue_duration_s":0.808908,"scheduling_latency_s":0.808925,"redis_calls":71,"redis_duration_s":0.018261,"redis_read_bytes":120,"redis_write_bytes":42574,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002518,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001411,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.01361,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000722,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033234,"duration_s":0.122715,"completed_at":"2026-09-08T18:40:01.308Z","load_balancing_strategy":"primary","db_duration_s":0.002772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1244gitlab # [ 240.642950] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5a30095b205c9472bf69be18","created_at":"2026-09-08T18:40:00.396Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71ba7dbe19f5614d9d5ac6a22691c864","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.404Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5a30095b205c9472bf69be18: start","job_status":"start","queue_duration_s":0.904838,"scheduling_latency_s":0.904855}1245gitlab # [ 240.656154] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5a30095b205c9472bf69be18","created_at":"2026-09-08T18:40:00.396Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71ba7dbe19f5614d9d5ac6a22691c864","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.404Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5a30095b205c9472bf69be18: done: 0.032446 sec","job_status":"done","queue_duration_s":0.904838,"scheduling_latency_s":0.904855,"redis_calls":12,"redis_duration_s":0.00312,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055386,"exclusive_lock_hold_duration_s":0.01069434,"duration_s":0.032446,"completed_at":"2026-09-08T18:40:01.342Z","load_balancing_strategy":"primary","db_duration_s":0.005449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1246gitlab # [ 240.696402] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e444a3ec21d0b23617b90328","created_at":"2026-09-08T18:40:00.422Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fddc85db374f29cb69323f3d81d5c7f9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.431Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e444a3ec21d0b23617b90328: start","job_status":"start","queue_duration_s":0.912326,"scheduling_latency_s":0.912344}1247gitlab # [ 240.710369] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.356Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1248gitlab # [ 240.713553] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.364Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e444a3ec21d0b23617b90328","created_at":"2026-09-08T18:40:00.422Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fddc85db374f29cb69323f3d81d5c7f9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.431Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e444a3ec21d0b23617b90328: done: 0.020644 sec","job_status":"done","queue_duration_s":0.912326,"scheduling_latency_s":0.912344,"redis_calls":10,"redis_duration_s":0.00236,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000503,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032684,"duration_s":0.020644,"completed_at":"2026-09-08T18:40:01.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1249gitlab # [ 240.754665] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ca36b836d29fb9eadbaaf506","created_at":"2026-09-08T18:40:00.444Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd07bf7c3ecdfb2b704cd877db080d8c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.450Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ca36b836d29fb9eadbaaf506: start","job_status":"start","queue_duration_s":0.91482,"scheduling_latency_s":0.914836}1250gitlab # [ 240.768427] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ca36b836d29fb9eadbaaf506","created_at":"2026-09-08T18:40:00.444Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd07bf7c3ecdfb2b704cd877db080d8c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.450Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ca36b836d29fb9eadbaaf506: done: 0.029561 sec","job_status":"done","queue_duration_s":0.91482,"scheduling_latency_s":0.914836,"redis_calls":10,"redis_duration_s":0.003246,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000595,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036138,"duration_s":0.029561,"completed_at":"2026-09-08T18:40:01.395Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004694,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1251gitlab # [ 240.810036] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"8a122b5b5c48bb824549ee0b","created_at":"2026-09-08T18:40:00.454Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c991717e7928fb76467a65155f01b0ca","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.471Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-8a122b5b5c48bb824549ee0b: start","job_status":"start","queue_duration_s":0.92475,"scheduling_latency_s":0.924767}1252gitlab # [ 241.041776] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"8a122b5b5c48bb824549ee0b","created_at":"2026-09-08T18:40:00.454Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c991717e7928fb76467a65155f01b0ca","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.471Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-8a122b5b5c48bb824549ee0b: done: 0.414574 sec","job_status":"done","queue_duration_s":0.92475,"scheduling_latency_s":0.924767,"redis_calls":565,"redis_duration_s":0.09601299999999999,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001413,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.094049,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.340996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038642,"duration_s":0.414574,"completed_at":"2026-09-08T18:40:01.811Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1253gitlab # [ 241.116375] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"edcbc92eb6685277a4896aa1","created_at":"2026-09-08T18:40:00.493Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3352ce74e82802c61b1ad870974b93ba","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.533Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-edcbc92eb6685277a4896aa1: start","job_status":"start","queue_duration_s":1.279872,"scheduling_latency_s":1.279897}1254gitlab # [ 241.132373] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.828Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1255gitlab # [ 241.136372] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"edcbc92eb6685277a4896aa1","created_at":"2026-09-08T18:40:00.493Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3352ce74e82802c61b1ad870974b93ba","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.533Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-edcbc92eb6685277a4896aa1: done: 0.023027 sec","job_status":"done","queue_duration_s":1.279872,"scheduling_latency_s":1.279897,"redis_calls":10,"redis_duration_s":0.0026520000000000003,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00059,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034156,"duration_s":0.023027,"completed_at":"2026-09-08T18:40:01.836Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1256gitlab # [ 241.178916] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.838Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e4061e97d79912d5a69ad494","created_at":"2026-09-08T18:40:00.580Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9f9e221def9536995d20f00bdad09f4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T18:40:00.610Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e4061e97d79912d5a69ad494: start","job_status":"start","queue_duration_s":1.227556,"scheduling_latency_s":1.227576}1257gitlab # [ 241.193189] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.851Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1258gitlab # [ 241.196830] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e4061e97d79912d5a69ad494","created_at":"2026-09-08T18:40:00.580Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9f9e221def9536995d20f00bdad09f4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T18:40:00.610Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e4061e97d79912d5a69ad494: done: 0.021997 sec","job_status":"done","queue_duration_s":1.227556,"scheduling_latency_s":1.227576,"redis_calls":10,"redis_duration_s":0.002644,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000624,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003364,"duration_s":0.021997,"completed_at":"2026-09-08T18:40:01.860Z","load_balancing_strategy":"primary","db_duration_s":0.000769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1259gitlab # [ 241.236961] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.861Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a21ecfab405f920be20e099c","created_at":"2026-09-08T18:40:00.621Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a2baa4406657b21b544cccb2a7700d4","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.651Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a21ecfab405f920be20e099c: start","job_status":"start","queue_duration_s":1.209971,"scheduling_latency_s":1.209988}1260gitlab # [ 241.250264] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.917Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a21ecfab405f920be20e099c","created_at":"2026-09-08T18:40:00.621Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a2baa4406657b21b544cccb2a7700d4","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.651Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a21ecfab405f920be20e099c: done: 0.055953 sec","job_status":"done","queue_duration_s":1.209971,"scheduling_latency_s":1.209988,"redis_calls":10,"redis_duration_s":0.002876,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000568,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051686,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.055953,"completed_at":"2026-09-08T18:40:01.917Z","load_balancing_strategy":"primary","db_duration_s":0.029826,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1261gitlab # [ 241.291778] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:01.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"bc42e7818bc5cb96d62f9786","created_at":"2026-09-08T18:40:00.673Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"723698fa581b7e2c26a36ace8a70da93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.697Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-bc42e7818bc5cb96d62f9786: start","job_status":"start","queue_duration_s":1.221569,"scheduling_latency_s":1.221592}1262gitlab # [ 241.457658] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"bc42e7818bc5cb96d62f9786","created_at":"2026-09-08T18:40:00.673Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"723698fa581b7e2c26a36ace8a70da93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.697Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-bc42e7818bc5cb96d62f9786: done: 0.307627 sec","job_status":"done","queue_duration_s":1.221569,"scheduling_latency_s":1.221592,"redis_calls":14,"redis_duration_s":0.004016,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001246,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001297,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.231765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070734,"exclusive_lock_hold_duration_s":0.27927778,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.307627,"completed_at":"2026-09-08T18:40:02.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.083155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1263gitlab # [ 241.551775] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.228Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1d55ef84ae8ed5a7964bfd84","created_at":"2026-09-08T18:40:00.757Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ddba1a26db96a7c0454094f47854f69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.788Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1d55ef84ae8ed5a7964bfd84: start","job_status":"start","queue_duration_s":1.439918,"scheduling_latency_s":1.439936}1264gitlab # [ 241.569002] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1d55ef84ae8ed5a7964bfd84","created_at":"2026-09-08T18:40:00.757Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ddba1a26db96a7c0454094f47854f69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.788Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1d55ef84ae8ed5a7964bfd84: done: 0.065491 sec","job_status":"done","queue_duration_s":1.439918,"scheduling_latency_s":1.439936,"redis_calls":14,"redis_duration_s":0.0033,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001147,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001201,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00066936,"exclusive_lock_hold_duration_s":0.04013012,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.065491,"completed_at":"2026-09-08T18:40:02.294Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1265gitlab # [ 241.614310] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a9929cab576e5981697b8496","created_at":"2026-09-08T18:40:00.802Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cb5db4d2d037e80a9ca0d0ae661160f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.822Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a9929cab576e5981697b8496: start","job_status":"start","queue_duration_s":1.473796,"scheduling_latency_s":1.473815}1266gitlab # [ 241.628750] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.307Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1267gitlab # [ 241.631988] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a9929cab576e5981697b8496","created_at":"2026-09-08T18:40:00.802Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cb5db4d2d037e80a9ca0d0ae661160f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.822Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a9929cab576e5981697b8496: done: 0.020127 sec","job_status":"done","queue_duration_s":1.473796,"scheduling_latency_s":1.473815,"redis_calls":10,"redis_duration_s":0.001349,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025592,"duration_s":0.020127,"completed_at":"2026-09-08T18:40:02.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1268gitlab # [ 241.672980] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"1cf3070751fffd1b62853a9b","created_at":"2026-09-08T18:40:00.876Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"426c6a5a0d77d12b1021057c1193d7dc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.906Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-1cf3070751fffd1b62853a9b: start","job_status":"start","queue_duration_s":1.41103,"scheduling_latency_s":1.411048}1269gitlab # [ 241.687167] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.366Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"1cf3070751fffd1b62853a9b","created_at":"2026-09-08T18:40:00.876Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"426c6a5a0d77d12b1021057c1193d7dc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:00.906Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-1cf3070751fffd1b62853a9b: done: 0.049814 sec","job_status":"done","queue_duration_s":1.41103,"scheduling_latency_s":1.411048,"redis_calls":12,"redis_duration_s":0.001936,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000332,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055434,"exclusive_lock_hold_duration_s":0.02056986,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.049814,"completed_at":"2026-09-08T18:40:02.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 241.730358] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.368Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9078169f7456d830ea3db2d5","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9078169f7456d830ea3db2d5: start","job_status":"start","queue_duration_s":1.357713,"scheduling_latency_s":1.357733}1271gitlab # [ 241.742270] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.386Z","message":"No existing merge request to be cleaned up."}1272gitlab # [ 241.743822] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.402Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9078169f7456d830ea3db2d5","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9078169f7456d830ea3db2d5: done: 0.03381 sec","job_status":"done","queue_duration_s":1.357713,"scheduling_latency_s":1.357733,"redis_calls":13,"redis_duration_s":0.00203,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024846,"duration_s":0.03381,"completed_at":"2026-09-08T18:40:02.402Z","load_balancing_strategy":"primary","db_duration_s":0.005658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1273gitlab # [ 241.782899] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.403Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2207b410c61e0927c819a0e2","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2207b410c61e0927c819a0e2: start","job_status":"start","queue_duration_s":1.392673,"scheduling_latency_s":1.392698}1274gitlab # [ 241.794826] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.414Z","message":"No existing merge request to be cleaned up."}1275gitlab # [ 241.796509] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.423Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2207b410c61e0927c819a0e2","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2207b410c61e0927c819a0e2: done: 0.020472 sec","job_status":"done","queue_duration_s":1.392673,"scheduling_latency_s":1.392698,"redis_calls":10,"redis_duration_s":0.0013909999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000781,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020472,"completed_at":"2026-09-08T18:40:02.423Z","load_balancing_strategy":"primary","db_duration_s":0.002446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 241.834913] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.425Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9245909e9dcbac55d0236139","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9245909e9dcbac55d0236139: start","job_status":"start","queue_duration_s":1.414596,"scheduling_latency_s":1.414618}1277gitlab # [ 241.847080] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.436Z","message":"No existing merge request to be cleaned up."}1278gitlab # [ 241.848759] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.445Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9245909e9dcbac55d0236139","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9245909e9dcbac55d0236139: done: 0.020102 sec","job_status":"done","queue_duration_s":1.414596,"scheduling_latency_s":1.414618,"redis_calls":10,"redis_duration_s":0.001413,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020102,"completed_at":"2026-09-08T18:40:02.445Z","load_balancing_strategy":"primary","db_duration_s":0.00241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1279gitlab # [ 241.886054] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.446Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4d74a6bf6586ce4b3944be1d","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4d74a6bf6586ce4b3944be1d: start","job_status":"start","queue_duration_s":1.435853,"scheduling_latency_s":1.435872}1280gitlab # [ 241.898024] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.457Z","message":"No existing merge request to be cleaned up."}1281gitlab # [ 241.899646] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.465Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4d74a6bf6586ce4b3944be1d","created_at":"2026-09-08T18:40:00.903Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-df50376634e1442b","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"fb36a5e0b038d2c35c27d62562df63f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.010Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4d74a6bf6586ce4b3944be1d: done: 0.019483 sec","job_status":"done","queue_duration_s":1.435853,"scheduling_latency_s":1.435872,"redis_calls":10,"redis_duration_s":0.001357,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019483,"completed_at":"2026-09-08T18:40:02.465Z","load_balancing_strategy":"primary","db_duration_s":0.002198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 241.936814] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a5a9575a2dc2a06d274c638","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a5a9575a2dc2a06d274c638: start","job_status":"start","queue_duration_s":1.169403,"scheduling_latency_s":1.169421}1283gitlab # [ 241.949342] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a5a9575a2dc2a06d274c638","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a5a9575a2dc2a06d274c638: done: 0.022957 sec","job_status":"done","queue_duration_s":1.169403,"scheduling_latency_s":1.169421,"redis_calls":16,"redis_duration_s":0.002015,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024158,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022957,"completed_at":"2026-09-08T18:40:02.490Z","load_balancing_strategy":"primary","db_duration_s":0.000571,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1284gitlab # [ 241.989232] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"870db565cc7d6a824fd3db7b","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-870db565cc7d6a824fd3db7b: start","job_status":"start","queue_duration_s":1.193592,"scheduling_latency_s":1.193611}1285gitlab # [ 242.001244] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"870db565cc7d6a824fd3db7b","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-870db565cc7d6a824fd3db7b: done: 0.015805 sec","job_status":"done","queue_duration_s":1.193592,"scheduling_latency_s":1.193611,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015805,"completed_at":"2026-09-08T18:40:02.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 242.040216] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9b8d5e2e8589a97fac54f0e","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9b8d5e2e8589a97fac54f0e: start","job_status":"start","queue_duration_s":1.210485,"scheduling_latency_s":1.210504}1287gitlab # [ 242.051993] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9b8d5e2e8589a97fac54f0e","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9b8d5e2e8589a97fac54f0e: done: 0.015497 sec","job_status":"done","queue_duration_s":1.210485,"scheduling_latency_s":1.210504,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015497,"completed_at":"2026-09-08T18:40:02.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 242.090745] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19c5d25caaed667115279417","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19c5d25caaed667115279417: start","job_status":"start","queue_duration_s":1.22706,"scheduling_latency_s":1.227078}1289gitlab # [ 242.102918] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19c5d25caaed667115279417","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19c5d25caaed667115279417: done: 0.015489 sec","job_status":"done","queue_duration_s":1.22706,"scheduling_latency_s":1.227078,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015489,"completed_at":"2026-09-08T18:40:02.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 242.140770] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e241572951bf27110bc4f82","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e241572951bf27110bc4f82: start","job_status":"start","queue_duration_s":1.243781,"scheduling_latency_s":1.2438}1291gitlab # [ 242.152895] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e241572951bf27110bc4f82","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e241572951bf27110bc4f82: done: 0.015095 sec","job_status":"done","queue_duration_s":1.243781,"scheduling_latency_s":1.2438,"redis_calls":13,"redis_duration_s":0.001404,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015095,"completed_at":"2026-09-08T18:40:02.556Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 242.190643] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e41ee9224d55f3d0f615ea62","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e41ee9224d55f3d0f615ea62: start","job_status":"start","queue_duration_s":1.259956,"scheduling_latency_s":1.259974}1293gitlab # [ 242.202774] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e41ee9224d55f3d0f615ea62","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e41ee9224d55f3d0f615ea62: done: 0.015272 sec","job_status":"done","queue_duration_s":1.259956,"scheduling_latency_s":1.259974,"redis_calls":13,"redis_duration_s":0.001466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015272,"completed_at":"2026-09-08T18:40:02.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 242.241483] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e52b8a11ee33d072a8f768cf","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e52b8a11ee33d072a8f768cf: start","job_status":"start","queue_duration_s":1.276603,"scheduling_latency_s":1.276622}1295gitlab # [ 242.253325] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e52b8a11ee33d072a8f768cf","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e52b8a11ee33d072a8f768cf: done: 0.015596 sec","job_status":"done","queue_duration_s":1.276603,"scheduling_latency_s":1.276622,"redis_calls":13,"redis_duration_s":0.001716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015596,"completed_at":"2026-09-08T18:40:02.589Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 242.291794] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e03d1da66be8ea94237799cd","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e03d1da66be8ea94237799cd: start","job_status":"start","queue_duration_s":1.29364,"scheduling_latency_s":1.293658}1297gitlab # [ 242.304114] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e03d1da66be8ea94237799cd","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e03d1da66be8ea94237799cd: done: 0.014965 sec","job_status":"done","queue_duration_s":1.29364,"scheduling_latency_s":1.293658,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014965,"completed_at":"2026-09-08T18:40:02.606Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 242.341917] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f172252f6356876630b33453","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f172252f6356876630b33453: start","job_status":"start","queue_duration_s":1.30986,"scheduling_latency_s":1.309879}1299gitlab # [ 242.354120] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f172252f6356876630b33453","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f172252f6356876630b33453: done: 0.015359 sec","job_status":"done","queue_duration_s":1.30986,"scheduling_latency_s":1.309879,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015359,"completed_at":"2026-09-08T18:40:02.622Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 242.392409] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29896ab2d94e62bfba5a4671","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29896ab2d94e62bfba5a4671: start","job_status":"start","queue_duration_s":1.326469,"scheduling_latency_s":1.326487}1301gitlab # [ 242.404259] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29896ab2d94e62bfba5a4671","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29896ab2d94e62bfba5a4671: done: 0.014995 sec","job_status":"done","queue_duration_s":1.326469,"scheduling_latency_s":1.326487,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014995,"completed_at":"2026-09-08T18:40:02.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 242.443371] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"705f7d5ca2151f886ed7e1f1","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-705f7d5ca2151f886ed7e1f1: start","job_status":"start","queue_duration_s":1.34273,"scheduling_latency_s":1.342749}1303gitlab # [ 242.455667] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"705f7d5ca2151f886ed7e1f1","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-705f7d5ca2151f886ed7e1f1: done: 0.01634 sec","job_status":"done","queue_duration_s":1.34273,"scheduling_latency_s":1.342749,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01634,"completed_at":"2026-09-08T18:40:02.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 242.493918] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb79e7bba7673a5308b0b17d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb79e7bba7673a5308b0b17d: start","job_status":"start","queue_duration_s":1.360165,"scheduling_latency_s":1.360184}1305gitlab # [ 242.506228] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb79e7bba7673a5308b0b17d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb79e7bba7673a5308b0b17d: done: 0.01532 sec","job_status":"done","queue_duration_s":1.360165,"scheduling_latency_s":1.360184,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01532,"completed_at":"2026-09-08T18:40:02.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 242.544255] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1518922c13277abdb70d8010","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1518922c13277abdb70d8010: start","job_status":"start","queue_duration_s":1.376562,"scheduling_latency_s":1.37658}1307gitlab # [ 242.556238] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1518922c13277abdb70d8010","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1518922c13277abdb70d8010: done: 0.015468 sec","job_status":"done","queue_duration_s":1.376562,"scheduling_latency_s":1.37658,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015468,"completed_at":"2026-09-08T18:40:02.689Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 242.593854] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573f9d01678de3c2804a3e92","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573f9d01678de3c2804a3e92: start","job_status":"start","queue_duration_s":1.393097,"scheduling_latency_s":1.393115}1309gitlab # [ 242.605977] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573f9d01678de3c2804a3e92","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573f9d01678de3c2804a3e92: done: 0.015576 sec","job_status":"done","queue_duration_s":1.393097,"scheduling_latency_s":1.393115,"redis_calls":13,"redis_duration_s":0.00167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015576,"completed_at":"2026-09-08T18:40:02.706Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 242.644873] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"197e1510a4658d551062aa2c","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-197e1510a4658d551062aa2c: start","job_status":"start","queue_duration_s":1.409826,"scheduling_latency_s":1.409843}1311gitlab # [ 242.656721] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"197e1510a4658d551062aa2c","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-197e1510a4658d551062aa2c: done: 0.015206 sec","job_status":"done","queue_duration_s":1.409826,"scheduling_latency_s":1.409843,"redis_calls":13,"redis_duration_s":0.0014190000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015206,"completed_at":"2026-09-08T18:40:02.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 242.694965] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9696975ec3d131d4a56a1e0d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9696975ec3d131d4a56a1e0d: start","job_status":"start","queue_duration_s":1.426091,"scheduling_latency_s":1.42611}1313gitlab # [ 242.707207] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9696975ec3d131d4a56a1e0d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9696975ec3d131d4a56a1e0d: done: 0.014954 sec","job_status":"done","queue_duration_s":1.426091,"scheduling_latency_s":1.42611,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014954,"completed_at":"2026-09-08T18:40:02.738Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 242.745410] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12510ea9f33bd1e15fb538c9","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12510ea9f33bd1e15fb538c9: start","job_status":"start","queue_duration_s":1.44227,"scheduling_latency_s":1.442287}1315gitlab # [ 242.757624] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12510ea9f33bd1e15fb538c9","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12510ea9f33bd1e15fb538c9: done: 0.015389 sec","job_status":"done","queue_duration_s":1.44227,"scheduling_latency_s":1.442287,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015389,"completed_at":"2026-09-08T18:40:02.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 242.795692] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3fd7824cbaa2b5506f1a4cd","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3fd7824cbaa2b5506f1a4cd: start","job_status":"start","queue_duration_s":1.45921,"scheduling_latency_s":1.459229}1317gitlab # [ 242.807903] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3fd7824cbaa2b5506f1a4cd","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3fd7824cbaa2b5506f1a4cd: done: 0.015052 sec","job_status":"done","queue_duration_s":1.45921,"scheduling_latency_s":1.459229,"redis_calls":13,"redis_duration_s":0.001429,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015052,"completed_at":"2026-09-08T18:40:02.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 242.847259] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51d64d2e2405f6b42b673e76","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51d64d2e2405f6b42b673e76: start","job_status":"start","queue_duration_s":1.475544,"scheduling_latency_s":1.475562}1319gitlab # [ 242.859363] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51d64d2e2405f6b42b673e76","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51d64d2e2405f6b42b673e76: done: 0.017146 sec","job_status":"done","queue_duration_s":1.475544,"scheduling_latency_s":1.475562,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017146,"completed_at":"2026-09-08T18:40:02.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 242.897747] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2893bf4ddf95452f3a2755c","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2893bf4ddf95452f3a2755c: start","job_status":"start","queue_duration_s":1.494053,"scheduling_latency_s":1.494073}1321gitlab # [ 242.909896] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2893bf4ddf95452f3a2755c","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2893bf4ddf95452f3a2755c: done: 0.015933 sec","job_status":"done","queue_duration_s":1.494053,"scheduling_latency_s":1.494073,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015933,"completed_at":"2026-09-08T18:40:02.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 242.948290] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ebfb022788cc71342d3c734","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ebfb022788cc71342d3c734: start","job_status":"start","queue_duration_s":1.511428,"scheduling_latency_s":1.511447}1323gitlab # [ 242.960362] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ebfb022788cc71342d3c734","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ebfb022788cc71342d3c734: done: 0.015707 sec","job_status":"done","queue_duration_s":1.511428,"scheduling_latency_s":1.511447,"redis_calls":13,"redis_duration_s":0.001775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015707,"completed_at":"2026-09-08T18:40:02.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 242.998554] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc844f1f43f517a510743086","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc844f1f43f517a510743086: start","job_status":"start","queue_duration_s":1.528191,"scheduling_latency_s":1.528208}1325gitlab # [ 243.010782] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc844f1f43f517a510743086","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc844f1f43f517a510743086: done: 0.015719 sec","job_status":"done","queue_duration_s":1.528191,"scheduling_latency_s":1.528208,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015719,"completed_at":"2026-09-08T18:40:02.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 243.049812] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc185ed9098c250d7199cf86","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc185ed9098c250d7199cf86: start","job_status":"start","queue_duration_s":1.545148,"scheduling_latency_s":1.545166}1327gitlab # [ 243.062360] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc185ed9098c250d7199cf86","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc185ed9098c250d7199cf86: done: 0.015278 sec","job_status":"done","queue_duration_s":1.545148,"scheduling_latency_s":1.545166,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015278,"completed_at":"2026-09-08T18:40:02.858Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 243.101742] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae6acf576cdd35f09e7d1aec","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae6acf576cdd35f09e7d1aec: start","job_status":"start","queue_duration_s":1.561503,"scheduling_latency_s":1.561521}1329gitlab # [ 243.114159] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae6acf576cdd35f09e7d1aec","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae6acf576cdd35f09e7d1aec: done: 0.015676 sec","job_status":"done","queue_duration_s":1.561503,"scheduling_latency_s":1.561521,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015676,"completed_at":"2026-09-08T18:40:02.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 243.153491] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5804f777962f7a7db48a3ee","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5804f777962f7a7db48a3ee: start","job_status":"start","queue_duration_s":1.578285,"scheduling_latency_s":1.578303}1331gitlab # [ 243.165770] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5804f777962f7a7db48a3ee","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5804f777962f7a7db48a3ee: done: 0.015721 sec","job_status":"done","queue_duration_s":1.578285,"scheduling_latency_s":1.578303,"redis_calls":13,"redis_duration_s":0.0016979999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015721,"completed_at":"2026-09-08T18:40:02.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 243.204346] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84394dfa7bf6e02a8e7ffdf8","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84394dfa7bf6e02a8e7ffdf8: start","job_status":"start","queue_duration_s":1.595074,"scheduling_latency_s":1.595092}1333gitlab # [ 243.216691] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84394dfa7bf6e02a8e7ffdf8","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84394dfa7bf6e02a8e7ffdf8: done: 0.015081 sec","job_status":"done","queue_duration_s":1.595074,"scheduling_latency_s":1.595092,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015081,"completed_at":"2026-09-08T18:40:02.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 243.256424] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9e34b45c8e7a34f420d41fa","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9e34b45c8e7a34f420d41fa: start","job_status":"start","queue_duration_s":1.611489,"scheduling_latency_s":1.611507}1335gitlab # [ 243.268481] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9e34b45c8e7a34f420d41fa","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9e34b45c8e7a34f420d41fa: done: 0.01545 sec","job_status":"done","queue_duration_s":1.611489,"scheduling_latency_s":1.611507,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01545,"completed_at":"2026-09-08T18:40:02.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 243.307674] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2782ee62b9e50d5bda9daa69","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2782ee62b9e50d5bda9daa69: start","job_status":"start","queue_duration_s":1.628173,"scheduling_latency_s":1.628191}1337gitlab # [ 243.320105] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2782ee62b9e50d5bda9daa69","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2782ee62b9e50d5bda9daa69: done: 0.014985 sec","job_status":"done","queue_duration_s":1.628173,"scheduling_latency_s":1.628191,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014985,"completed_at":"2026-09-08T18:40:02.940Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 243.359294] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1498f936eb2ea301db2d8055","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1498f936eb2ea301db2d8055: start","job_status":"start","queue_duration_s":1.644399,"scheduling_latency_s":1.644416}1339gitlab # [ 243.371825] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1498f936eb2ea301db2d8055","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1498f936eb2ea301db2d8055: done: 0.015305 sec","job_status":"done","queue_duration_s":1.644399,"scheduling_latency_s":1.644416,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015305,"completed_at":"2026-09-08T18:40:02.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 243.411514] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8ad3d64fb1c65b51ffaabf2","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8ad3d64fb1c65b51ffaabf2: start","job_status":"start","queue_duration_s":1.661061,"scheduling_latency_s":1.66108}1341gitlab # [ 243.424812] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8ad3d64fb1c65b51ffaabf2","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8ad3d64fb1c65b51ffaabf2: done: 0.015263 sec","job_status":"done","queue_duration_s":1.661061,"scheduling_latency_s":1.66108,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015263,"completed_at":"2026-09-08T18:40:02.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 243.464478] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d65b33aa9bc8712add95cc0d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d65b33aa9bc8712add95cc0d: start","job_status":"start","queue_duration_s":1.677743,"scheduling_latency_s":1.677761}1343gitlab # [ 243.476855] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d65b33aa9bc8712add95cc0d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d65b33aa9bc8712add95cc0d: done: 0.015429 sec","job_status":"done","queue_duration_s":1.677743,"scheduling_latency_s":1.677761,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015429,"completed_at":"2026-09-08T18:40:02.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 243.515601] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:02.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2eb2f4976178d78c26f449ed","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2eb2f4976178d78c26f449ed: start","job_status":"start","queue_duration_s":1.694264,"scheduling_latency_s":1.694284}1345gitlab # [ 243.528176] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2eb2f4976178d78c26f449ed","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2eb2f4976178d78c26f449ed: done: 0.015637 sec","job_status":"done","queue_duration_s":1.694264,"scheduling_latency_s":1.694284,"redis_calls":13,"redis_duration_s":0.0017230000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015637,"completed_at":"2026-09-08T18:40:03.007Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 243.567050] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5faa3ea18f8c8f1d5f9593be","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5faa3ea18f8c8f1d5f9593be: start","job_status":"start","queue_duration_s":1.711011,"scheduling_latency_s":1.71103}1347gitlab # [ 243.579554] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5faa3ea18f8c8f1d5f9593be","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5faa3ea18f8c8f1d5f9593be: done: 0.015575 sec","job_status":"done","queue_duration_s":1.711011,"scheduling_latency_s":1.71103,"redis_calls":13,"redis_duration_s":0.001768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015575,"completed_at":"2026-09-08T18:40:03.024Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 243.618103] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf85b7cc365a4d71ee9d9010","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf85b7cc365a4d71ee9d9010: start","job_status":"start","queue_duration_s":1.727946,"scheduling_latency_s":1.727964}1349gitlab # [ 243.630497] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf85b7cc365a4d71ee9d9010","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf85b7cc365a4d71ee9d9010: done: 0.015436 sec","job_status":"done","queue_duration_s":1.727946,"scheduling_latency_s":1.727964,"redis_calls":13,"redis_duration_s":0.001688,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015436,"completed_at":"2026-09-08T18:40:03.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 243.669730] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ce3bb4f6757d901e186ed5c","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ce3bb4f6757d901e186ed5c: start","job_status":"start","queue_duration_s":1.7445,"scheduling_latency_s":1.744518}1351gitlab # [ 243.681785] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ce3bb4f6757d901e186ed5c","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ce3bb4f6757d901e186ed5c: done: 0.015371 sec","job_status":"done","queue_duration_s":1.7445,"scheduling_latency_s":1.744518,"redis_calls":13,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015371,"completed_at":"2026-09-08T18:40:03.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 243.720576] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdcbdd54a68c6d49c29e9e82","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdcbdd54a68c6d49c29e9e82: start","job_status":"start","queue_duration_s":1.760911,"scheduling_latency_s":1.76093}1353gitlab # [ 243.732685] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdcbdd54a68c6d49c29e9e82","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdcbdd54a68c6d49c29e9e82: done: 0.015084 sec","job_status":"done","queue_duration_s":1.760911,"scheduling_latency_s":1.76093,"redis_calls":13,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015084,"completed_at":"2026-09-08T18:40:03.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 243.771329] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d3ae5f289f2c963471ad53f","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d3ae5f289f2c963471ad53f: start","job_status":"start","queue_duration_s":1.777303,"scheduling_latency_s":1.777321}1355gitlab # [ 243.783620] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d3ae5f289f2c963471ad53f","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d3ae5f289f2c963471ad53f: done: 0.01576 sec","job_status":"done","queue_duration_s":1.777303,"scheduling_latency_s":1.777321,"redis_calls":13,"redis_duration_s":0.001448,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01576,"completed_at":"2026-09-08T18:40:03.090Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 243.822681] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0cfb6f876e03dbd87b2aeeb","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0cfb6f876e03dbd87b2aeeb: start","job_status":"start","queue_duration_s":1.794309,"scheduling_latency_s":1.794328}1357gitlab # [ 243.861133] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0cfb6f876e03dbd87b2aeeb","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0cfb6f876e03dbd87b2aeeb: done: 0.015183 sec","job_status":"done","queue_duration_s":1.794309,"scheduling_latency_s":1.794328,"redis_calls":13,"redis_duration_s":0.001433,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015183,"completed_at":"2026-09-08T18:40:03.107Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 243.960543] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7953900dceea051edb038feb","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7953900dceea051edb038feb: start","job_status":"start","queue_duration_s":1.810815,"scheduling_latency_s":1.810832}1359gitlab # [ 243.986370] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7953900dceea051edb038feb","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7953900dceea051edb038feb: done: 0.015206 sec","job_status":"done","queue_duration_s":1.810815,"scheduling_latency_s":1.810832,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015206,"completed_at":"2026-09-08T18:40:03.123Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 244.037049] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70e582485221ebe2dd709480","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70e582485221ebe2dd709480: start","job_status":"start","queue_duration_s":1.827269,"scheduling_latency_s":1.827288}1361gitlab # [ 244.049779] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70e582485221ebe2dd709480","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70e582485221ebe2dd709480: done: 0.015373 sec","job_status":"done","queue_duration_s":1.827269,"scheduling_latency_s":1.827288,"redis_calls":13,"redis_duration_s":0.001471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015373,"completed_at":"2026-09-08T18:40:03.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 244.089231] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d72dff04f7d34f1d71164825","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d72dff04f7d34f1d71164825: start","job_status":"start","queue_duration_s":1.843989,"scheduling_latency_s":1.844006}1363gitlab # [ 244.101537] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d72dff04f7d34f1d71164825","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d72dff04f7d34f1d71164825: done: 0.0152 sec","job_status":"done","queue_duration_s":1.843989,"scheduling_latency_s":1.844006,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0152,"completed_at":"2026-09-08T18:40:03.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 244.140151] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f930484ccea467d61b4aca3b","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f930484ccea467d61b4aca3b: start","job_status":"start","queue_duration_s":1.860253,"scheduling_latency_s":1.86027}1365gitlab # [ 244.152705] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f930484ccea467d61b4aca3b","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f930484ccea467d61b4aca3b: done: 0.016604 sec","job_status":"done","queue_duration_s":1.860253,"scheduling_latency_s":1.86027,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016604,"completed_at":"2026-09-08T18:40:03.174Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 244.191316] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea29e6d96576f76d969a2f16","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea29e6d96576f76d969a2f16: start","job_status":"start","queue_duration_s":1.87798,"scheduling_latency_s":1.877997}1367gitlab # [ 244.203994] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea29e6d96576f76d969a2f16","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea29e6d96576f76d969a2f16: done: 0.01552 sec","job_status":"done","queue_duration_s":1.87798,"scheduling_latency_s":1.877997,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01552,"completed_at":"2026-09-08T18:40:03.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 244.242054] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8dad86a12ed162ffa0a09b7","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8dad86a12ed162ffa0a09b7: start","job_status":"start","queue_duration_s":1.894741,"scheduling_latency_s":1.894758}1369gitlab # [ 244.254250] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8dad86a12ed162ffa0a09b7","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8dad86a12ed162ffa0a09b7: done: 0.015859 sec","job_status":"done","queue_duration_s":1.894741,"scheduling_latency_s":1.894758,"redis_calls":13,"redis_duration_s":0.001534,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015859,"completed_at":"2026-09-08T18:40:03.208Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 244.293086] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98cd4e6207a40c37324bc9a1","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98cd4e6207a40c37324bc9a1: start","job_status":"start","queue_duration_s":1.911715,"scheduling_latency_s":1.911733}1371gitlab # [ 244.305271] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98cd4e6207a40c37324bc9a1","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98cd4e6207a40c37324bc9a1: done: 0.015376 sec","job_status":"done","queue_duration_s":1.911715,"scheduling_latency_s":1.911733,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015376,"completed_at":"2026-09-08T18:40:03.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 244.343808] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889ba9d0e8d711c0284c1bba","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889ba9d0e8d711c0284c1bba: start","job_status":"start","queue_duration_s":1.928151,"scheduling_latency_s":1.928169}1373gitlab # [ 244.356126] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889ba9d0e8d711c0284c1bba","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889ba9d0e8d711c0284c1bba: done: 0.016836 sec","job_status":"done","queue_duration_s":1.928151,"scheduling_latency_s":1.928169,"redis_calls":13,"redis_duration_s":0.001538,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016836,"completed_at":"2026-09-08T18:40:03.242Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 244.393960] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c837a906e676cfcf3378d22f","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c837a906e676cfcf3378d22f: start","job_status":"start","queue_duration_s":1.946316,"scheduling_latency_s":1.946335}1375gitlab # [ 244.406175] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c837a906e676cfcf3378d22f","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c837a906e676cfcf3378d22f: done: 0.015086 sec","job_status":"done","queue_duration_s":1.946316,"scheduling_latency_s":1.946335,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015086,"completed_at":"2026-09-08T18:40:03.259Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 244.444151] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20663fc48c0d507883574bc4","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20663fc48c0d507883574bc4: start","job_status":"start","queue_duration_s":1.96261,"scheduling_latency_s":1.962643}1377gitlab # [ 244.455942] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20663fc48c0d507883574bc4","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20663fc48c0d507883574bc4: done: 0.01511 sec","job_status":"done","queue_duration_s":1.96261,"scheduling_latency_s":1.962643,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01511,"completed_at":"2026-09-08T18:40:03.275Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 244.495155] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4831283c76ff92a5a279373b","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4831283c76ff92a5a279373b: start","job_status":"start","queue_duration_s":1.979001,"scheduling_latency_s":1.979018}1379gitlab # [ 244.507383] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4831283c76ff92a5a279373b","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4831283c76ff92a5a279373b: done: 0.016186 sec","job_status":"done","queue_duration_s":1.979001,"scheduling_latency_s":1.979018,"redis_calls":13,"redis_duration_s":0.0015500000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016186,"completed_at":"2026-09-08T18:40:03.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 244.545858] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d42bd26c39964bf51883e60d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d42bd26c39964bf51883e60d: start","job_status":"start","queue_duration_s":1.99647,"scheduling_latency_s":1.996489}1381gitlab # [ 244.557796] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:03.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d42bd26c39964bf51883e60d","created_at":"2026-09-08T18:40:01.209Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-0813f15a2c3241db","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"14943c7bfbc84bb4a6eeaa5998ba504f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:40:01.297Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d42bd26c39964bf51883e60d: done: 0.015401 sec","job_status":"done","queue_duration_s":1.99647,"scheduling_latency_s":1.996489,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015401,"completed_at":"2026-09-08T18:40:03.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 246.931343] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:07.699Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"58616fa1e01c3524259f00b5","created_at":"2026-09-08T18:40:07.677Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b8f986579cab95f91a6d8e801ebb0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.696Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-58616fa1e01c3524259f00b5: start","job_status":"start","queue_duration_s":0.003365,"scheduling_latency_s":0.003426}1383gitlab # [ 247.038702] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:07.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"58616fa1e01c3524259f00b5","created_at":"2026-09-08T18:40:07.677Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b8f986579cab95f91a6d8e801ebb0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.696Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-58616fa1e01c3524259f00b5: done: 0.107952 sec","job_status":"done","queue_duration_s":0.003365,"scheduling_latency_s":0.003426,"redis_calls":7,"redis_duration_s":0.023774,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01942,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004354,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.107952,"completed_at":"2026-09-08T18:40:07.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 247.128153] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:07.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"e2e025f49c1edee65fee4a86","created_at":"2026-09-08T18:40:07.769Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae8cd32d95d8ef41ddb6ed454c79bc42","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.800Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-e2e025f49c1edee65fee4a86: start","job_status":"start","queue_duration_s":0.076444,"scheduling_latency_s":0.076502}1385gitlab # [ 247.257335] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.025Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"e2e025f49c1edee65fee4a86","created_at":"2026-09-08T18:40:07.769Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae8cd32d95d8ef41ddb6ed454c79bc42","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.800Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-e2e025f49c1edee65fee4a86: done: 0.148937 sec","job_status":"done","queue_duration_s":0.076444,"scheduling_latency_s":0.076502,"redis_calls":10,"redis_duration_s":0.039749,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007126,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.019007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.013616,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00159696,"duration_s":0.148937,"completed_at":"2026-09-08T18:40:08.025Z","load_balancing_strategy":"primary","db_duration_s":0.002264,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 247.355801] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"59ca3f1e2c25e9abf3d06c68","created_at":"2026-09-08T18:40:07.870Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0989592402d970e72a2176c95de2597","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.899Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-59ca3f1e2c25e9abf3d06c68: start","job_status":"start","queue_duration_s":0.19769,"scheduling_latency_s":0.197745}1387gitlab # [ 247.675830] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"59ca3f1e2c25e9abf3d06c68","created_at":"2026-09-08T18:40:07.870Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0989592402d970e72a2176c95de2597","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.899Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-59ca3f1e2c25e9abf3d06c68: done: 0.347716 sec","job_status":"done","queue_duration_s":0.19769,"scheduling_latency_s":0.197745,"redis_calls":10,"redis_duration_s":0.013224,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002118,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00736,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003746,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00203278,"duration_s":0.347716,"completed_at":"2026-09-08T18:40:08.444Z","load_balancing_strategy":"primary","db_duration_s":0.030011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 247.774904] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.450Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fa4e46a2d97dd786b7a34956","created_at":"2026-09-08T18:40:07.906Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6aef8f50ddd683ff25c3ea8b92b25004","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.914Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fa4e46a2d97dd786b7a34956: start","job_status":"start","queue_duration_s":0.536115,"scheduling_latency_s":0.536167}1389gitlab # [ 247.832914] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.601Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fa4e46a2d97dd786b7a34956","created_at":"2026-09-08T18:40:07.906Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6aef8f50ddd683ff25c3ea8b92b25004","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.914Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fa4e46a2d97dd786b7a34956: done: 0.151768 sec","job_status":"done","queue_duration_s":0.536115,"scheduling_latency_s":0.536167,"redis_calls":7,"redis_duration_s":0.002736,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000756,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.151768,"completed_at":"2026-09-08T18:40:08.601Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 247.892891] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"83d6b43f36b533da6c28ca48","created_at":"2026-09-08T18:40:07.924Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"472261006c049cdaab3037a1232f604c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.943Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-83d6b43f36b533da6c28ca48: start","job_status":"start","queue_duration_s":0.66184,"scheduling_latency_s":0.661889}1391gitlab # [ 247.935446] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"83d6b43f36b533da6c28ca48","created_at":"2026-09-08T18:40:07.924Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"472261006c049cdaab3037a1232f604c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:07.943Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-83d6b43f36b533da6c28ca48: done: 0.040047 sec","job_status":"done","queue_duration_s":0.66184,"scheduling_latency_s":0.661889,"redis_calls":10,"redis_duration_s":0.0026420000000000003,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001507,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00933692,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.040047,"completed_at":"2026-09-08T18:40:08.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003584,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1392gitlab # [ 248.010871] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"8ef35b48e3362a06d5bd498e","created_at":"2026-09-08T18:40:08.016Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3473b41ef93ff816e96d6d8917b66013","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.032Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-8ef35b48e3362a06d5bd498e: start","job_status":"start","queue_duration_s":0.614642,"scheduling_latency_s":0.614669}1393gitlab # [ 248.026701] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"8ef35b48e3362a06d5bd498e","created_at":"2026-09-08T18:40:08.016Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3473b41ef93ff816e96d6d8917b66013","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.032Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-8ef35b48e3362a06d5bd498e: done: 0.04942 sec","job_status":"done","queue_duration_s":0.614642,"scheduling_latency_s":0.614669,"redis_calls":10,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030634,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00376486},"duration_s":0.04942,"completed_at":"2026-09-08T18:40:08.696Z","load_balancing_strategy":"primary","db_duration_s":0.012272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 248.069891] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"a27fe0f214304a97dcce175e","created_at":"2026-09-08T18:40:08.079Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1183bdb53a4950d0e4c6be07d20a7c95","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.088Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-a27fe0f214304a97dcce175e: start","job_status":"start","queue_duration_s":0.609677,"scheduling_latency_s":0.609695}1395gitlab # [ 248.084008] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"a27fe0f214304a97dcce175e","created_at":"2026-09-08T18:40:08.079Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1183bdb53a4950d0e4c6be07d20a7c95","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.088Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-a27fe0f214304a97dcce175e: done: 0.026081 sec","job_status":"done","queue_duration_s":0.609677,"scheduling_latency_s":0.609695,"redis_calls":13,"redis_duration_s":0.001638,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000659,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023006,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.026081,"completed_at":"2026-09-08T18:40:08.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 248.127637] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.725Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"88c593dd8feda272fcdeb2fd","created_at":"2026-09-08T18:40:08.096Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"878504de3272a8f0a2a5f4cdd670bff3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.113Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-88c593dd8feda272fcdeb2fd: start","job_status":"start","queue_duration_s":0.611857,"scheduling_latency_s":0.611875}1397gitlab # [ 248.141910] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"88c593dd8feda272fcdeb2fd","created_at":"2026-09-08T18:40:08.096Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"878504de3272a8f0a2a5f4cdd670bff3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.113Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-88c593dd8feda272fcdeb2fd: done: 0.033164 sec","job_status":"done","queue_duration_s":0.611857,"scheduling_latency_s":0.611875,"redis_calls":20,"redis_duration_s":0.002552,"redis_read_bytes":21,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000791,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000467,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022426,"duration_s":0.033164,"completed_at":"2026-09-08T18:40:08.758Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 248.185512] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2cd191f4881e95c1f9487dcd","created_at":"2026-09-08T18:40:08.123Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10bc8be87e9ca5192d8e1c53e0cee581","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.131Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2cd191f4881e95c1f9487dcd: start","job_status":"start","queue_duration_s":0.628371,"scheduling_latency_s":0.628388}1399gitlab # [ 248.199786] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.788Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2cd191f4881e95c1f9487dcd","created_at":"2026-09-08T18:40:08.123Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10bc8be87e9ca5192d8e1c53e0cee581","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.131Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2cd191f4881e95c1f9487dcd: done: 0.028686 sec","job_status":"done","queue_duration_s":0.628371,"scheduling_latency_s":0.628388,"redis_calls":10,"redis_duration_s":0.001331,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023558,"duration_s":0.028686,"completed_at":"2026-09-08T18:40:08.788Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 248.241354] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ef5d0892839117da5fa0350e","created_at":"2026-09-08T18:40:08.135Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66825f5df189e9006b0260408041ff8b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.147Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ef5d0892839117da5fa0350e: start","job_status":"start","queue_duration_s":0.641746,"scheduling_latency_s":0.641764}1401gitlab # [ 248.255282] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ef5d0892839117da5fa0350e","created_at":"2026-09-08T18:40:08.135Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66825f5df189e9006b0260408041ff8b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.147Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ef5d0892839117da5fa0350e: done: 0.025308 sec","job_status":"done","queue_duration_s":0.641746,"scheduling_latency_s":0.641764,"redis_calls":12,"redis_duration_s":0.001585,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000611,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004572,"exclusive_lock_hold_duration_s":0.0064399,"duration_s":0.025308,"completed_at":"2026-09-08T18:40:08.814Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003963,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 248.296823] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.815Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"80085c8aec85c632217e223b","created_at":"2026-09-08T18:40:08.296Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"8ab52aab50beb700745ad49a7506b025","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.315Z","job_size_bytes":3,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-80085c8aec85c632217e223b: start","job_status":"start","queue_duration_s":0.500229,"scheduling_latency_s":0.500246}1403gitlab # [ 248.310226] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.834Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"80085c8aec85c632217e223b","created_at":"2026-09-08T18:40:08.296Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"8ab52aab50beb700745ad49a7506b025","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.315Z","job_size_bytes":3,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-80085c8aec85c632217e223b: done: 0.018635 sec","job_status":"done","queue_duration_s":0.500229,"scheduling_latency_s":0.500246,"redis_calls":10,"redis_duration_s":0.001246,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002206,"duration_s":0.018635,"completed_at":"2026-09-08T18:40:08.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 248.351021] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"676929167b2deaf32ad7f160","created_at":"2026-09-08T18:40:08.331Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b10c8a79ceba109166bfac2f93496efd","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.349Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-676929167b2deaf32ad7f160: start","job_status":"start","queue_duration_s":0.485946,"scheduling_latency_s":0.485962}1405gitlab # [ 248.365171] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"676929167b2deaf32ad7f160","created_at":"2026-09-08T18:40:08.331Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b10c8a79ceba109166bfac2f93496efd","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.349Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-676929167b2deaf32ad7f160: done: 0.030228 sec","job_status":"done","queue_duration_s":0.485946,"scheduling_latency_s":0.485962,"redis_calls":10,"redis_duration_s":0.001363,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026508,"duration_s":0.030228,"completed_at":"2026-09-08T18:40:08.865Z","load_balancing_strategy":"primary","db_duration_s":0.005966,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 248.405696] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"b6ae14894f364d70213c3ac2","created_at":"2026-09-08T18:40:08.371Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea6132a1e868bf5963b1872bc8e40aed","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.387Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-b6ae14894f364d70213c3ac2: start","job_status":"start","queue_duration_s":0.479874,"scheduling_latency_s":0.479893}1407gitlab # [ 248.419736] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"b6ae14894f364d70213c3ac2","created_at":"2026-09-08T18:40:08.371Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea6132a1e868bf5963b1872bc8e40aed","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.387Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-b6ae14894f364d70213c3ac2: done: 0.021817 sec","job_status":"done","queue_duration_s":0.479874,"scheduling_latency_s":0.479893,"redis_calls":11,"redis_duration_s":0.001605,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000321,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00024,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022334,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.021817,"completed_at":"2026-09-08T18:40:08.888Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000715,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 248.462796] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.890Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"3ff90786eb29ebd4c14c3164","created_at":"2026-09-08T18:40:08.446Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a600d834f39386585f0f846c31f9759","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.458Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-3ff90786eb29ebd4c14c3164: start","job_status":"start","queue_duration_s":0.431655,"scheduling_latency_s":0.431672}1409gitlab # [ 248.475668] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.909Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"3ff90786eb29ebd4c14c3164","created_at":"2026-09-08T18:40:08.446Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a600d834f39386585f0f846c31f9759","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.458Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-3ff90786eb29ebd4c14c3164: done: 0.019777 sec","job_status":"done","queue_duration_s":0.431655,"scheduling_latency_s":0.431672,"redis_calls":12,"redis_duration_s":0.0022890000000000002,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000365,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001397,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021758,"duration_s":0.019777,"completed_at":"2026-09-08T18:40:08.909Z","load_balancing_strategy":"primary","db_duration_s":0.000784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 248.517319] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"178bb187273b83bf813707d4","created_at":"2026-09-08T18:40:08.515Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02c1bf14d133372aeaa8e6db992df2d5","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.525Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-178bb187273b83bf813707d4: start","job_status":"start","queue_duration_s":0.385265,"scheduling_latency_s":0.385282}1411gitlab # [ 248.530090] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.933Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"178bb187273b83bf813707d4","created_at":"2026-09-08T18:40:08.515Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02c1bf14d133372aeaa8e6db992df2d5","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.525Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-178bb187273b83bf813707d4: done: 0.022583 sec","job_status":"done","queue_duration_s":0.385265,"scheduling_latency_s":0.385282,"redis_calls":10,"redis_duration_s":0.002033,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001468,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022526,"duration_s":0.022583,"completed_at":"2026-09-08T18:40:08.933Z","load_balancing_strategy":"primary","db_duration_s":0.000715,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 248.571016] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.934Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"2095b08316690c3b0f09b12f","created_at":"2026-09-08T18:40:08.737Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e934a392d5964cb1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"878504de3272a8f0a2a5f4cdd670bff3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.747Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-2095b08316690c3b0f09b12f: start","job_status":"start","queue_duration_s":0.187358,"scheduling_latency_s":0.187375}1413gitlab # [ 248.585620] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.959Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"2095b08316690c3b0f09b12f","created_at":"2026-09-08T18:40:08.737Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e934a392d5964cb1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"878504de3272a8f0a2a5f4cdd670bff3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.747Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-2095b08316690c3b0f09b12f: done: 0.024896 sec","job_status":"done","queue_duration_s":0.187358,"scheduling_latency_s":0.187375,"redis_calls":10,"redis_duration_s":0.0013239999999999999,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000274,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021744,"duration_s":0.024896,"completed_at":"2026-09-08T18:40:08.959Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 248.627819] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.960Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"a6b1624a5ae7063765c9bfd4","created_at":"2026-09-08T18:40:08.747Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e934a392d5964cb1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"878504de3272a8f0a2a5f4cdd670bff3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.752Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-a6b1624a5ae7063765c9bfd4: start","job_status":"start","queue_duration_s":0.208488,"scheduling_latency_s":0.208506}1415gitlab # [ 248.643132] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:40:08.983Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"a6b1624a5ae7063765c9bfd4","created_at":"2026-09-08T18:40:08.747Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e934a392d5964cb1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"878504de3272a8f0a2a5f4cdd670bff3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-08T18:40:08.752Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-a6b1624a5ae7063765c9bfd4: done: 0.022779 sec","job_status":"done","queue_duration_s":0.208488,"scheduling_latency_s":0.208506,"redis_calls":10,"redis_duration_s":0.001391,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022762,"duration_s":0.022779,"completed_at":"2026-09-08T18:40:08.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 270.833590] setup-gitlab-start[2949]: W, [2026-09-08T18:40:31.602531 #2949] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1417gitlab # [ 300.874128] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:01.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"00c705a1228799fd2f9a20fc","created_at":"2026-09-08T18:41:01.483Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1aa9991eaef992279b9d4a054a7666ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.639Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-00c705a1228799fd2f9a20fc: start","job_status":"start","queue_duration_s":0.00298,"scheduling_latency_s":0.00304}1418gitlab # [ 300.962653] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:01.731Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"00c705a1228799fd2f9a20fc","created_at":"2026-09-08T18:41:01.483Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1aa9991eaef992279b9d4a054a7666ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.639Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-00c705a1228799fd2f9a20fc: done: 0.08937 sec","job_status":"done","queue_duration_s":0.00298,"scheduling_latency_s":0.00304,"redis_calls":16,"redis_duration_s":0.011344,"redis_read_bytes":1157,"redis_write_bytes":1478,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003988,"redis_feature_flag_read_bytes":1152,"redis_feature_flag_write_bytes":647,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005972,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001384,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08937,"completed_at":"2026-09-08T18:41:01.731Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 301.045847] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:01.737Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1d6e88b081c0d2b5bea9e484","created_at":"2026-09-08T18:41:01.690Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4ca4f0de58f4eb4d336785fb206f183","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.707Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1d6e88b081c0d2b5bea9e484: start","job_status":"start","queue_duration_s":0.030022,"scheduling_latency_s":0.030071}1420gitlab # [ 301.194560] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:01.963Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1d6e88b081c0d2b5bea9e484","created_at":"2026-09-08T18:41:01.690Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4ca4f0de58f4eb4d336785fb206f183","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.707Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1d6e88b081c0d2b5bea9e484: done: 0.225913 sec","job_status":"done","queue_duration_s":0.030022,"scheduling_latency_s":0.030071,"redis_calls":11,"redis_duration_s":0.009658,"redis_read_bytes":517,"redis_write_bytes":1011,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003738,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":300,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003028,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002892,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.225913,"completed_at":"2026-09-08T18:41:01.963Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.137856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 301.285580] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:01.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"15e9d74ec1860549d6302cfa","created_at":"2026-09-08T18:41:01.723Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13cd3a1fad0e1c9833a9135a5ca04e98","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.740Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-15e9d74ec1860549d6302cfa: start","job_status":"start","queue_duration_s":0.231477,"scheduling_latency_s":0.231531}1422gitlab # [ 301.349338] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"15e9d74ec1860549d6302cfa","created_at":"2026-09-08T18:41:01.723Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13cd3a1fad0e1c9833a9135a5ca04e98","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.740Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-15e9d74ec1860549d6302cfa: done: 0.146432 sec","job_status":"done","queue_duration_s":0.231477,"scheduling_latency_s":0.231531,"redis_calls":12,"redis_duration_s":0.041985,"redis_read_bytes":264,"redis_write_bytes":1388,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.021512,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":149,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010249,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.010224,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03944604,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.146432,"completed_at":"2026-09-08T18:41:02.118Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011693,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1423gitlab # [ 301.422017] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5036a13594cbb75417534733","created_at":"2026-09-08T18:41:01.816Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e773b73e896638484fded92a55ec58f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.826Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5036a13594cbb75417534733: start","job_status":"start","queue_duration_s":0.306669,"scheduling_latency_s":0.306721}1424gitlab # [ 301.540218] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5036a13594cbb75417534733","created_at":"2026-09-08T18:41:01.816Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e773b73e896638484fded92a55ec58f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.826Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5036a13594cbb75417534733: done: 0.175138 sec","job_status":"done","queue_duration_s":0.306669,"scheduling_latency_s":0.306721,"redis_calls":8,"redis_duration_s":0.026587,"redis_read_bytes":133,"redis_write_bytes":879,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001042,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":103,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.022283,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.175138,"completed_at":"2026-09-08T18:41:02.308Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 301.629907] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5614b0e45416e8936b748083","created_at":"2026-09-08T18:41:01.837Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.844Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5614b0e45416e8936b748083: start","job_status":"start","queue_duration_s":0.475121,"scheduling_latency_s":0.475161}1426gitlab # [ 301.662149] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5614b0e45416e8936b748083","created_at":"2026-09-08T18:41:01.837Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.844Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5614b0e45416e8936b748083: done: 0.111722 sec","job_status":"done","queue_duration_s":0.475121,"scheduling_latency_s":0.475161,"redis_calls":16,"redis_duration_s":0.002356,"redis_read_bytes":272,"redis_write_bytes":4221,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":199,"redis_queues_calls":2,"redis_queues_duration_s":0.000272,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001368,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.111722,"completed_at":"2026-09-08T18:41:02.430Z","load_balancing_strategy":"primary","db_duration_s":0.001533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 301.729754] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"12054c4d55444ddeac1056f3","created_at":"2026-09-08T18:41:01.849Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e02a98cb8a5477e3784197222481e39","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.855Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-12054c4d55444ddeac1056f3: start","job_status":"start","queue_duration_s":0.577552,"scheduling_latency_s":0.577581}1428gitlab # [ 301.745968] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"12054c4d55444ddeac1056f3","created_at":"2026-09-08T18:41:01.849Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e02a98cb8a5477e3784197222481e39","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.855Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-12054c4d55444ddeac1056f3: done: 0.019759 sec","job_status":"done","queue_duration_s":0.577552,"scheduling_latency_s":0.577581,"redis_calls":10,"redis_duration_s":0.001539,"redis_read_bytes":135,"redis_write_bytes":1184,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000124,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":97,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052346,"exclusive_lock_hold_duration_s":0.0030177,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.019759,"completed_at":"2026-09-08T18:41:02.452Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 301.789002] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5180f21e33212dd9e18f8934","created_at":"2026-09-08T18:41:01.872Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"190cc11db6d16d538dee0fdd869c8d65","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T18:41:01.879Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5180f21e33212dd9e18f8934: start","job_status":"start","queue_duration_s":0.57444,"scheduling_latency_s":0.574458}1430gitlab # [ 301.802156] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5180f21e33212dd9e18f8934","created_at":"2026-09-08T18:41:01.872Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"190cc11db6d16d538dee0fdd869c8d65","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T18:41:01.879Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5180f21e33212dd9e18f8934: done: 0.022969 sec","job_status":"done","queue_duration_s":0.57444,"scheduling_latency_s":0.574458,"redis_calls":8,"redis_duration_s":0.001112,"redis_read_bytes":133,"redis_write_bytes":836,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000117,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022969,"completed_at":"2026-09-08T18:41:02.477Z","load_balancing_strategy":"primary","db_duration_s":0.00161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 301.842463] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.478Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"014908cc69cbb35bfab2ecbd","created_at":"2026-09-08T18:41:01.883Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"921ff93652aeaf74c06407543988178d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T18:41:01.886Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-014908cc69cbb35bfab2ecbd: start","job_status":"start","queue_duration_s":0.592025,"scheduling_latency_s":0.592042}1432gitlab # [ 301.855917] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.486Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1433gitlab # [ 301.859371] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"014908cc69cbb35bfab2ecbd","created_at":"2026-09-08T18:41:01.883Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"921ff93652aeaf74c06407543988178d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T18:41:01.886Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-014908cc69cbb35bfab2ecbd: done: 0.014216 sec","job_status":"done","queue_duration_s":0.592025,"scheduling_latency_s":0.592042,"redis_calls":8,"redis_duration_s":0.001098,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000115,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014216,"completed_at":"2026-09-08T18:41:02.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 301.902524] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"85a4ef0857dfd174fba9ab2c","created_at":"2026-09-08T18:41:01.902Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8be8e92affeefd4167d81d89216b2344","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.905Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-85a4ef0857dfd174fba9ab2c: start","job_status":"start","queue_duration_s":0.58792,"scheduling_latency_s":0.587936}1435gitlab # [ 301.916802] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.502Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1436gitlab # [ 301.920337] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"85a4ef0857dfd174fba9ab2c","created_at":"2026-09-08T18:41:01.902Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8be8e92affeefd4167d81d89216b2344","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.905Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-85a4ef0857dfd174fba9ab2c: done: 0.013477 sec","job_status":"done","queue_duration_s":0.58792,"scheduling_latency_s":0.587936,"redis_calls":8,"redis_duration_s":0.001774,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000112,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001415,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013477,"completed_at":"2026-09-08T18:41:02.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 301.962192] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2fea7b132f8d90e1d685cf00","created_at":"2026-09-08T18:41:01.912Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.927Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2fea7b132f8d90e1d685cf00: start","job_status":"start","queue_duration_s":0.581245,"scheduling_latency_s":0.581262}1438gitlab # [ 301.975731] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2fea7b132f8d90e1d685cf00","created_at":"2026-09-08T18:41:01.912Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.927Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2fea7b132f8d90e1d685cf00: done: 0.09942 sec","job_status":"done","queue_duration_s":0.581245,"scheduling_latency_s":0.581262,"redis_calls":65,"redis_duration_s":0.008988,"redis_read_bytes":748,"redis_write_bytes":41402,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.000566,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.000479,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.00755,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.09942,"completed_at":"2026-09-08T18:41:02.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 302.018445] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.609Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"139e01c0187949c83208e250","created_at":"2026-09-08T18:41:01.954Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9150a534250e8666b825f0f300498b0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.960Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-139e01c0187949c83208e250: start","job_status":"start","queue_duration_s":0.649275,"scheduling_latency_s":0.649293}1440gitlab # [ 302.032108] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"139e01c0187949c83208e250","created_at":"2026-09-08T18:41:01.954Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9150a534250e8666b825f0f300498b0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:01.960Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-139e01c0187949c83208e250: done: 0.019672 sec","job_status":"done","queue_duration_s":0.649275,"scheduling_latency_s":0.649293,"redis_calls":10,"redis_duration_s":0.001508,"redis_read_bytes":135,"redis_write_bytes":1167,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000131,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000661,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004665,"exclusive_lock_hold_duration_s":0.00510598,"duration_s":0.019672,"completed_at":"2026-09-08T18:41:02.629Z","load_balancing_strategy":"primary","db_duration_s":0.00252,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 302.074284] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"01de444e77de8ea0bc11e901","created_at":"2026-09-08T18:41:01.997Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"131210ec44412948ebb59d80e5c0b250","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.017Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-01de444e77de8ea0bc11e901: start","job_status":"start","queue_duration_s":0.613116,"scheduling_latency_s":0.613134}1442gitlab # [ 302.088600] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.639Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1443gitlab # [ 302.091730] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"01de444e77de8ea0bc11e901","created_at":"2026-09-08T18:41:01.997Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"131210ec44412948ebb59d80e5c0b250","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.017Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-01de444e77de8ea0bc11e901: done: 0.013793 sec","job_status":"done","queue_duration_s":0.613116,"scheduling_latency_s":0.613134,"redis_calls":8,"redis_duration_s":0.001036,"redis_read_bytes":133,"redis_write_bytes":873,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000111,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013793,"completed_at":"2026-09-08T18:41:02.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 302.133601] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d10a711a413b16e54e686a71","created_at":"2026-09-08T18:41:02.058Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd4df22be1842fcfa361e27f6643fef4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.071Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d10a711a413b16e54e686a71: start","job_status":"start","queue_duration_s":0.574258,"scheduling_latency_s":0.574274}1445gitlab # [ 302.147628] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.660Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d10a711a413b16e54e686a71","created_at":"2026-09-08T18:41:02.058Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd4df22be1842fcfa361e27f6643fef4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.071Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d10a711a413b16e54e686a71: done: 0.015473 sec","job_status":"done","queue_duration_s":0.574258,"scheduling_latency_s":0.574274,"redis_calls":8,"redis_duration_s":0.001067,"redis_read_bytes":133,"redis_write_bytes":764,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000125,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015473,"completed_at":"2026-09-08T18:41:02.660Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 302.189307] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"34c8c0ad7c54ba3e9558fcb5","created_at":"2026-09-08T18:41:02.090Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9298e3fd0886032d577b8b3af8f89f65","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.100Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-34c8c0ad7c54ba3e9558fcb5: start","job_status":"start","queue_duration_s":0.561708,"scheduling_latency_s":0.561725}1447gitlab # [ 302.205071] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"34c8c0ad7c54ba3e9558fcb5","created_at":"2026-09-08T18:41:02.090Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9298e3fd0886032d577b8b3af8f89f65","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.100Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-34c8c0ad7c54ba3e9558fcb5: done: 0.312944 sec","job_status":"done","queue_duration_s":0.561708,"scheduling_latency_s":0.561725,"redis_calls":563,"redis_duration_s":0.055917999999999995,"redis_read_bytes":688,"redis_write_bytes":48843,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000108,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.055066,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.290304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.312944,"completed_at":"2026-09-08T18:41:02.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 302.246446] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.976Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"50c0b518da27e035b4595b87","created_at":"2026-09-08T18:41:02.116Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ba67b8d39eca9601df3be2e3bcb64c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.126Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-50c0b518da27e035b4595b87: start","job_status":"start","queue_duration_s":0.849853,"scheduling_latency_s":0.849882}1449gitlab # [ 302.261376] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.986Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1450gitlab # [ 302.265183] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.992Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"50c0b518da27e035b4595b87","created_at":"2026-09-08T18:41:02.116Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ba67b8d39eca9601df3be2e3bcb64c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.126Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-50c0b518da27e035b4595b87: done: 0.016183 sec","job_status":"done","queue_duration_s":0.849853,"scheduling_latency_s":0.849882,"redis_calls":8,"redis_duration_s":0.00123,"redis_read_bytes":133,"redis_write_bytes":931,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000131,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016183,"completed_at":"2026-09-08T18:41:02.992Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 302.308544] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:02.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"ab615266171d8b2cd6829bf1","created_at":"2026-09-08T18:41:02.164Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7b5b04d57d3288ccc1decbf1f2e4033","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T18:41:02.170Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-ab615266171d8b2cd6829bf1: start","job_status":"start","queue_duration_s":0.823061,"scheduling_latency_s":0.823082}1452gitlab # [ 302.322083] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.003Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1453gitlab # [ 302.325882] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"ab615266171d8b2cd6829bf1","created_at":"2026-09-08T18:41:02.164Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7b5b04d57d3288ccc1decbf1f2e4033","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T18:41:02.170Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-ab615266171d8b2cd6829bf1: done: 0.016165 sec","job_status":"done","queue_duration_s":0.823061,"scheduling_latency_s":0.823082,"redis_calls":8,"redis_duration_s":0.001281,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000141,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016165,"completed_at":"2026-09-08T18:41:03.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 302.367129] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"29f31bc8d67b9b8d5268c07e","created_at":"2026-09-08T18:41:02.180Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e43358842c080a28de5f3bbc851eed1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.185Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-29f31bc8d67b9b8d5268c07e: start","job_status":"start","queue_duration_s":0.825728,"scheduling_latency_s":0.825746}1455gitlab # [ 302.381350] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"29f31bc8d67b9b8d5268c07e","created_at":"2026-09-08T18:41:02.180Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e43358842c080a28de5f3bbc851eed1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.185Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-29f31bc8d67b9b8d5268c07e: done: 0.019383 sec","job_status":"done","queue_duration_s":0.825728,"scheduling_latency_s":0.825746,"redis_calls":8,"redis_duration_s":0.001082,"redis_read_bytes":133,"redis_write_bytes":936,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000116,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019383,"completed_at":"2026-09-08T18:41:03.030Z","load_balancing_strategy":"primary","db_duration_s":0.003792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 302.424046] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"b344ec34fe6df48e7c473502","created_at":"2026-09-08T18:41:02.225Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0068c54f57d3fd7ec46d6cffe56979e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.241Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-b344ec34fe6df48e7c473502: start","job_status":"start","queue_duration_s":0.790871,"scheduling_latency_s":0.790889}1457gitlab # [ 302.574527] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"b344ec34fe6df48e7c473502","created_at":"2026-09-08T18:41:02.225Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0068c54f57d3fd7ec46d6cffe56979e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.241Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-b344ec34fe6df48e7c473502: done: 0.31201 sec","job_status":"done","queue_duration_s":0.790871,"scheduling_latency_s":0.790889,"redis_calls":11,"redis_duration_s":0.002551,"redis_read_bytes":263,"redis_write_bytes":1138,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000237,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":151,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001159,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.064,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.212904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049524,"exclusive_lock_hold_duration_s":0.29374538,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.31201,"completed_at":"2026-09-08T18:41:03.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.116691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 302.662909] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d250ef04c6c8b7b124fded96","created_at":"2026-09-08T18:41:02.300Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd751d1843392ed34150234c687e4c76","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.310Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d250ef04c6c8b7b124fded96: start","job_status":"start","queue_duration_s":1.034566,"scheduling_latency_s":1.034591}1459gitlab # [ 302.681422] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d250ef04c6c8b7b124fded96","created_at":"2026-09-08T18:41:02.300Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd751d1843392ed34150234c687e4c76","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.310Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d250ef04c6c8b7b124fded96: done: 0.047889 sec","job_status":"done","queue_duration_s":1.034566,"scheduling_latency_s":1.034591,"redis_calls":10,"redis_duration_s":0.002214,"redis_read_bytes":135,"redis_write_bytes":1267,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000135,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":105,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00107262,"exclusive_lock_hold_duration_s":0.0286991,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.047889,"completed_at":"2026-09-08T18:41:03.393Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 302.732097] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"59b06c2fb7718eb530687e64","created_at":"2026-09-08T18:41:02.323Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc3f1ac69a794309527a8936db35ed3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.332Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-59b06c2fb7718eb530687e64: start","job_status":"start","queue_duration_s":1.062927,"scheduling_latency_s":1.062948}1461gitlab # [ 302.747558] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.405Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1462gitlab # [ 302.751112] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"59b06c2fb7718eb530687e64","created_at":"2026-09-08T18:41:02.323Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc3f1ac69a794309527a8936db35ed3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.332Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-59b06c2fb7718eb530687e64: done: 0.017948 sec","job_status":"done","queue_duration_s":1.062927,"scheduling_latency_s":1.062948,"redis_calls":8,"redis_duration_s":0.002003,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000158,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017948,"completed_at":"2026-09-08T18:41:03.412Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 302.793462] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e5ce91815a64586617f68ff","created_at":"2026-09-08T18:41:02.351Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe029423501c4990023dd1722f4cda43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.357Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e5ce91815a64586617f68ff: start","job_status":"start","queue_duration_s":1.057072,"scheduling_latency_s":1.057098}1464gitlab # [ 302.808130] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e5ce91815a64586617f68ff","created_at":"2026-09-08T18:41:02.351Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-af6290fd38bb4d83","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe029423501c4990023dd1722f4cda43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.357Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e5ce91815a64586617f68ff: done: 0.054352 sec","job_status":"done","queue_duration_s":1.057072,"scheduling_latency_s":1.057098,"redis_calls":10,"redis_duration_s":0.002548,"redis_read_bytes":135,"redis_write_bytes":1275,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000167,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":106,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001287,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00112444,"exclusive_lock_hold_duration_s":0.02247702,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.054352,"completed_at":"2026-09-08T18:41:03.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022697,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 302.854064] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.471Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a9e6f91816416fe4a304af3f","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a9e6f91816416fe4a304af3f: start","job_status":"start","queue_duration_s":1.047189,"scheduling_latency_s":1.047217}1466gitlab # [ 302.866487] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.489Z","message":"No existing merge request to be cleaned up."}1467gitlab # [ 302.868128] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.500Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a9e6f91816416fe4a304af3f","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a9e6f91816416fe4a304af3f: done: 0.028964 sec","job_status":"done","queue_duration_s":1.047189,"scheduling_latency_s":1.047217,"redis_calls":11,"redis_duration_s":0.00216,"redis_read_bytes":136,"redis_write_bytes":1104,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000176,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":82,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001182,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028964,"completed_at":"2026-09-08T18:41:03.500Z","load_balancing_strategy":"primary","db_duration_s":0.003171,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 302.909558] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.501Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dcf53f85761fa21a63987589","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dcf53f85761fa21a63987589: start","job_status":"start","queue_duration_s":1.077322,"scheduling_latency_s":1.077341}1469gitlab # [ 302.921960] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.513Z","message":"No existing merge request to be cleaned up."}1470gitlab # [ 302.923641] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.521Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dcf53f85761fa21a63987589","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dcf53f85761fa21a63987589: done: 0.020161 sec","job_status":"done","queue_duration_s":1.077322,"scheduling_latency_s":1.077341,"redis_calls":10,"redis_duration_s":0.0013419999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020161,"completed_at":"2026-09-08T18:41:03.521Z","load_balancing_strategy":"primary","db_duration_s":0.002697,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 302.962627] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.522Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"242654d5003c5118d87a4b0a","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-242654d5003c5118d87a4b0a: start","job_status":"start","queue_duration_s":1.098638,"scheduling_latency_s":1.098656}1472gitlab # [ 302.975366] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.534Z","message":"No existing merge request to be cleaned up."}1473gitlab # [ 302.977135] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.543Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"242654d5003c5118d87a4b0a","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-242654d5003c5118d87a4b0a: done: 0.020726 sec","job_status":"done","queue_duration_s":1.098638,"scheduling_latency_s":1.098656,"redis_calls":10,"redis_duration_s":0.001327,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000749,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020726,"completed_at":"2026-09-08T18:41:03.543Z","load_balancing_strategy":"primary","db_duration_s":0.002518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 303.015600] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.544Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5fb61d15ecadd02755e30eac","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5fb61d15ecadd02755e30eac: start","job_status":"start","queue_duration_s":1.120492,"scheduling_latency_s":1.12051}1475gitlab # [ 303.028622] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.556Z","message":"No existing merge request to be cleaned up."}1476gitlab # [ 303.030373] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.564Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5fb61d15ecadd02755e30eac","created_at":"2026-09-08T18:41:02.412Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-91fc727c51ce49c1","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3ed080853d93da2f137aef6beeaae9d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.424Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5fb61d15ecadd02755e30eac: done: 0.019957 sec","job_status":"done","queue_duration_s":1.120492,"scheduling_latency_s":1.12051,"redis_calls":10,"redis_duration_s":0.001386,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00078,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019957,"completed_at":"2026-09-08T18:41:03.564Z","load_balancing_strategy":"primary","db_duration_s":0.002605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 303.068747] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07fdda33c52ba92145bc9ebb","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07fdda33c52ba92145bc9ebb: start","job_status":"start","queue_duration_s":0.964389,"scheduling_latency_s":0.964408}1478gitlab # [ 303.080925] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07fdda33c52ba92145bc9ebb","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07fdda33c52ba92145bc9ebb: done: 0.016533 sec","job_status":"done","queue_duration_s":0.964389,"scheduling_latency_s":0.964408,"redis_calls":14,"redis_duration_s":0.0016929999999999998,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000127,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016533,"completed_at":"2026-09-08T18:41:03.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 303.122709] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"110f318ed936f1e87edefb50","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-110f318ed936f1e87edefb50: start","job_status":"start","queue_duration_s":0.982545,"scheduling_latency_s":0.982562}1480gitlab # [ 303.135300] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"110f318ed936f1e87edefb50","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-110f318ed936f1e87edefb50: done: 0.015254 sec","job_status":"done","queue_duration_s":0.982545,"scheduling_latency_s":0.982562,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015254,"completed_at":"2026-09-08T18:41:03.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 303.174541] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c870ca9a4d82f269092d841d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c870ca9a4d82f269092d841d: start","job_status":"start","queue_duration_s":0.998961,"scheduling_latency_s":0.998978}1482gitlab # [ 303.187007] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c870ca9a4d82f269092d841d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c870ca9a4d82f269092d841d: done: 0.015055 sec","job_status":"done","queue_duration_s":0.998961,"scheduling_latency_s":0.998978,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015055,"completed_at":"2026-09-08T18:41:03.615Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 303.226114] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bc20bb666ea999bc776660d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bc20bb666ea999bc776660d: start","job_status":"start","queue_duration_s":1.01509,"scheduling_latency_s":1.015107}1484gitlab # [ 303.238497] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bc20bb666ea999bc776660d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bc20bb666ea999bc776660d: done: 0.014705 sec","job_status":"done","queue_duration_s":1.01509,"scheduling_latency_s":1.015107,"redis_calls":13,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014705,"completed_at":"2026-09-08T18:41:03.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 303.276854] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"334c9d497b8ef990944a3486","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-334c9d497b8ef990944a3486: start","job_status":"start","queue_duration_s":1.03089,"scheduling_latency_s":1.030908}1486gitlab # [ 303.290070] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"334c9d497b8ef990944a3486","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-334c9d497b8ef990944a3486: done: 0.01503 sec","job_status":"done","queue_duration_s":1.03089,"scheduling_latency_s":1.030908,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01503,"completed_at":"2026-09-08T18:41:03.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 303.328383] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03ed21f3c609d5714c629c70","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03ed21f3c609d5714c629c70: start","job_status":"start","queue_duration_s":1.047554,"scheduling_latency_s":1.04757}1488gitlab # [ 303.340271] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03ed21f3c609d5714c629c70","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03ed21f3c609d5714c629c70: done: 0.015049 sec","job_status":"done","queue_duration_s":1.047554,"scheduling_latency_s":1.04757,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015049,"completed_at":"2026-09-08T18:41:03.663Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 303.378465] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0e300d1cd6711aa4874a93a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0e300d1cd6711aa4874a93a: start","job_status":"start","queue_duration_s":1.063731,"scheduling_latency_s":1.063748}1490gitlab # [ 303.390811] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0e300d1cd6711aa4874a93a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0e300d1cd6711aa4874a93a: done: 0.015684 sec","job_status":"done","queue_duration_s":1.063731,"scheduling_latency_s":1.063748,"redis_calls":13,"redis_duration_s":0.0015669999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015684,"completed_at":"2026-09-08T18:41:03.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 303.429269] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8993ff2b84962d6e93ac833","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8993ff2b84962d6e93ac833: start","job_status":"start","queue_duration_s":1.080882,"scheduling_latency_s":1.0809}1492gitlab # [ 303.441438] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8993ff2b84962d6e93ac833","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8993ff2b84962d6e93ac833: done: 0.016005 sec","job_status":"done","queue_duration_s":1.080882,"scheduling_latency_s":1.0809,"redis_calls":13,"redis_duration_s":0.0016519999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016005,"completed_at":"2026-09-08T18:41:03.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 303.479991] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98decd68b0bae9fa73fd1ec8","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98decd68b0bae9fa73fd1ec8: start","job_status":"start","queue_duration_s":1.098006,"scheduling_latency_s":1.098024}1494gitlab # [ 303.492000] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98decd68b0bae9fa73fd1ec8","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98decd68b0bae9fa73fd1ec8: done: 0.016003 sec","job_status":"done","queue_duration_s":1.098006,"scheduling_latency_s":1.098024,"redis_calls":13,"redis_duration_s":0.001644,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016003,"completed_at":"2026-09-08T18:41:03.715Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 303.531434] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfefe0692d237b81fc6d03bf","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfefe0692d237b81fc6d03bf: start","job_status":"start","queue_duration_s":1.115107,"scheduling_latency_s":1.115124}1496gitlab # [ 303.543629] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfefe0692d237b81fc6d03bf","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfefe0692d237b81fc6d03bf: done: 0.014598 sec","job_status":"done","queue_duration_s":1.115107,"scheduling_latency_s":1.115124,"redis_calls":13,"redis_duration_s":0.0013939999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014598,"completed_at":"2026-09-08T18:41:03.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 303.582265] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23b30bc0f5a59de5508093d9","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23b30bc0f5a59de5508093d9: start","job_status":"start","queue_duration_s":1.130805,"scheduling_latency_s":1.130822}1498gitlab # [ 303.594483] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23b30bc0f5a59de5508093d9","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23b30bc0f5a59de5508093d9: done: 0.015055 sec","job_status":"done","queue_duration_s":1.130805,"scheduling_latency_s":1.130822,"redis_calls":13,"redis_duration_s":0.0018909999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015055,"completed_at":"2026-09-08T18:41:03.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 303.632929] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92819195b3cf5f1b13043dcb","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92819195b3cf5f1b13043dcb: start","job_status":"start","queue_duration_s":1.146986,"scheduling_latency_s":1.147003}1500gitlab # [ 303.644947] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92819195b3cf5f1b13043dcb","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92819195b3cf5f1b13043dcb: done: 0.01454 sec","job_status":"done","queue_duration_s":1.146986,"scheduling_latency_s":1.147003,"redis_calls":13,"redis_duration_s":0.001377,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000875,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01454,"completed_at":"2026-09-08T18:41:03.762Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 303.683277] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e463ddfeefc3a461b77cc67","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e463ddfeefc3a461b77cc67: start","job_status":"start","queue_duration_s":1.162592,"scheduling_latency_s":1.16263}1502gitlab # [ 303.695447] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e463ddfeefc3a461b77cc67","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e463ddfeefc3a461b77cc67: done: 0.014948 sec","job_status":"done","queue_duration_s":1.162592,"scheduling_latency_s":1.16263,"redis_calls":13,"redis_duration_s":0.001379,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014948,"completed_at":"2026-09-08T18:41:03.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 303.734835] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c682bf108da3534fed8b8d2f","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c682bf108da3534fed8b8d2f: start","job_status":"start","queue_duration_s":1.178586,"scheduling_latency_s":1.178616}1504gitlab # [ 303.746980] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c682bf108da3534fed8b8d2f","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c682bf108da3534fed8b8d2f: done: 0.014798 sec","job_status":"done","queue_duration_s":1.178586,"scheduling_latency_s":1.178616,"redis_calls":13,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014798,"completed_at":"2026-09-08T18:41:03.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 303.785335] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2880e77180ae0101b98b0b87","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2880e77180ae0101b98b0b87: start","job_status":"start","queue_duration_s":1.194432,"scheduling_latency_s":1.194449}1506gitlab # [ 303.797387] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2880e77180ae0101b98b0b87","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2880e77180ae0101b98b0b87: done: 0.015482 sec","job_status":"done","queue_duration_s":1.194432,"scheduling_latency_s":1.194449,"redis_calls":13,"redis_duration_s":0.001389,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015482,"completed_at":"2026-09-08T18:41:03.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 303.836546] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf579cb71e6543f3d1606b3a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf579cb71e6543f3d1606b3a: start","job_status":"start","queue_duration_s":1.210995,"scheduling_latency_s":1.211012}1508gitlab # [ 303.874279] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf579cb71e6543f3d1606b3a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf579cb71e6543f3d1606b3a: done: 0.014323 sec","job_status":"done","queue_duration_s":1.210995,"scheduling_latency_s":1.211012,"redis_calls":13,"redis_duration_s":0.001378,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014323,"completed_at":"2026-09-08T18:41:03.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 303.973245] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86d422aa3980f06273a2fd9a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86d422aa3980f06273a2fd9a: start","job_status":"start","queue_duration_s":1.226429,"scheduling_latency_s":1.226447}1510gitlab # [ 303.997007] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86d422aa3980f06273a2fd9a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86d422aa3980f06273a2fd9a: done: 0.014849 sec","job_status":"done","queue_duration_s":1.226429,"scheduling_latency_s":1.226447,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014849,"completed_at":"2026-09-08T18:41:03.842Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 304.052108] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889264c46504dd8f3d49b9dd","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889264c46504dd8f3d49b9dd: start","job_status":"start","queue_duration_s":1.242349,"scheduling_latency_s":1.242366}1512gitlab # [ 304.066091] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889264c46504dd8f3d49b9dd","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889264c46504dd8f3d49b9dd: done: 0.017762 sec","job_status":"done","queue_duration_s":1.242349,"scheduling_latency_s":1.242366,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017762,"completed_at":"2026-09-08T18:41:03.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 304.105949] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e93f7cd62379842fe0602576","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e93f7cd62379842fe0602576: start","job_status":"start","queue_duration_s":1.261191,"scheduling_latency_s":1.261208}1514gitlab # [ 304.118346] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e93f7cd62379842fe0602576","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e93f7cd62379842fe0602576: done: 0.01468 sec","job_status":"done","queue_duration_s":1.261191,"scheduling_latency_s":1.261208,"redis_calls":13,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01468,"completed_at":"2026-09-08T18:41:03.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 304.158022] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b6dd49470e03ceeba32fd0b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b6dd49470e03ceeba32fd0b: start","job_status":"start","queue_duration_s":1.276931,"scheduling_latency_s":1.276965}1516gitlab # [ 304.170362] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b6dd49470e03ceeba32fd0b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b6dd49470e03ceeba32fd0b: done: 0.014377 sec","job_status":"done","queue_duration_s":1.276931,"scheduling_latency_s":1.276965,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014377,"completed_at":"2026-09-08T18:41:03.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 304.208822] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17f665ae937c65c16c544d0f","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17f665ae937c65c16c544d0f: start","job_status":"start","queue_duration_s":1.292349,"scheduling_latency_s":1.292365}1518gitlab # [ 304.220855] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17f665ae937c65c16c544d0f","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17f665ae937c65c16c544d0f: done: 0.015125 sec","job_status":"done","queue_duration_s":1.292349,"scheduling_latency_s":1.292365,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015125,"completed_at":"2026-09-08T18:41:03.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 304.259593] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fff188821bcbe6aa15016cae","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fff188821bcbe6aa15016cae: start","job_status":"start","queue_duration_s":1.315258,"scheduling_latency_s":1.315277}1520gitlab # [ 304.272062] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fff188821bcbe6aa15016cae","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fff188821bcbe6aa15016cae: done: 0.013928 sec","job_status":"done","queue_duration_s":1.315258,"scheduling_latency_s":1.315277,"redis_calls":13,"redis_duration_s":0.001359,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013928,"completed_at":"2026-09-08T18:41:03.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 304.310602] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c2adffb1cbabfbfe6096749","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c2adffb1cbabfbfe6096749: start","job_status":"start","queue_duration_s":1.330228,"scheduling_latency_s":1.330245}1522gitlab # [ 304.323136] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c2adffb1cbabfbfe6096749","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c2adffb1cbabfbfe6096749: done: 0.014131 sec","job_status":"done","queue_duration_s":1.330228,"scheduling_latency_s":1.330245,"redis_calls":13,"redis_duration_s":0.001606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014131,"completed_at":"2026-09-08T18:41:03.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 304.363153] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53bc52621bc97d1cb0b3ec0b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53bc52621bc97d1cb0b3ec0b: start","job_status":"start","queue_duration_s":1.34541,"scheduling_latency_s":1.345426}1524gitlab # [ 304.375749] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53bc52621bc97d1cb0b3ec0b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53bc52621bc97d1cb0b3ec0b: done: 0.013961 sec","job_status":"done","queue_duration_s":1.34541,"scheduling_latency_s":1.345426,"redis_calls":13,"redis_duration_s":0.001433,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013961,"completed_at":"2026-09-08T18:41:03.960Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 304.414570] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb7ebb5b2a71304ee8ec4f0b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb7ebb5b2a71304ee8ec4f0b: start","job_status":"start","queue_duration_s":1.360404,"scheduling_latency_s":1.36042}1526gitlab # [ 304.426649] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb7ebb5b2a71304ee8ec4f0b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb7ebb5b2a71304ee8ec4f0b: done: 0.013729 sec","job_status":"done","queue_duration_s":1.360404,"scheduling_latency_s":1.36042,"redis_calls":13,"redis_duration_s":0.001358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013729,"completed_at":"2026-09-08T18:41:03.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 304.464572] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15fea44f1690c0b2befbad85","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15fea44f1690c0b2befbad85: start","job_status":"start","queue_duration_s":1.375178,"scheduling_latency_s":1.375195}1528gitlab # [ 304.476486] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15fea44f1690c0b2befbad85","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15fea44f1690c0b2befbad85: done: 0.013739 sec","job_status":"done","queue_duration_s":1.375178,"scheduling_latency_s":1.375195,"redis_calls":13,"redis_duration_s":0.001376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000866,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013739,"completed_at":"2026-09-08T18:41:03.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 304.514614] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:03.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7138bf37f24b2d3b48b3cedd","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7138bf37f24b2d3b48b3cedd: start","job_status":"start","queue_duration_s":1.389961,"scheduling_latency_s":1.389978}1530gitlab # [ 304.526616] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7138bf37f24b2d3b48b3cedd","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7138bf37f24b2d3b48b3cedd: done: 0.014334 sec","job_status":"done","queue_duration_s":1.389961,"scheduling_latency_s":1.389978,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014334,"completed_at":"2026-09-08T18:41:04.005Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 304.565807] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc27c43b1376105cb09a991b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc27c43b1376105cb09a991b: start","job_status":"start","queue_duration_s":1.405429,"scheduling_latency_s":1.405447}1532gitlab # [ 304.578129] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc27c43b1376105cb09a991b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc27c43b1376105cb09a991b: done: 0.013796 sec","job_status":"done","queue_duration_s":1.405429,"scheduling_latency_s":1.405447,"redis_calls":13,"redis_duration_s":0.001388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013796,"completed_at":"2026-09-08T18:41:04.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 304.616209] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3c3865809a2b033b11e8b6b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3c3865809a2b033b11e8b6b: start","job_status":"start","queue_duration_s":1.420264,"scheduling_latency_s":1.420282}1534gitlab # [ 304.628161] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3c3865809a2b033b11e8b6b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3c3865809a2b033b11e8b6b: done: 0.013829 sec","job_status":"done","queue_duration_s":1.420264,"scheduling_latency_s":1.420282,"redis_calls":13,"redis_duration_s":0.001351,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013829,"completed_at":"2026-09-08T18:41:04.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 304.666140] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f32b0866eecb2297281a66","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f32b0866eecb2297281a66: start","job_status":"start","queue_duration_s":1.435167,"scheduling_latency_s":1.435185}1536gitlab # [ 304.678381] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f32b0866eecb2297281a66","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f32b0866eecb2297281a66: done: 0.01438 sec","job_status":"done","queue_duration_s":1.435167,"scheduling_latency_s":1.435185,"redis_calls":13,"redis_duration_s":0.001408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01438,"completed_at":"2026-09-08T18:41:04.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 304.716086] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51914d2d62e0dc86b4ec3e09","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51914d2d62e0dc86b4ec3e09: start","job_status":"start","queue_duration_s":1.450588,"scheduling_latency_s":1.450619}1538gitlab # [ 304.727976] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51914d2d62e0dc86b4ec3e09","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51914d2d62e0dc86b4ec3e09: done: 0.014095 sec","job_status":"done","queue_duration_s":1.450588,"scheduling_latency_s":1.450619,"redis_calls":13,"redis_duration_s":0.00139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014095,"completed_at":"2026-09-08T18:41:04.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 304.767103] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"814c1a45da0f8390f8987bae","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-814c1a45da0f8390f8987bae: start","job_status":"start","queue_duration_s":1.465744,"scheduling_latency_s":1.465763}1540gitlab # [ 304.779168] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"814c1a45da0f8390f8987bae","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-814c1a45da0f8390f8987bae: done: 0.014041 sec","job_status":"done","queue_duration_s":1.465744,"scheduling_latency_s":1.465763,"redis_calls":13,"redis_duration_s":0.001383,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014041,"completed_at":"2026-09-08T18:41:04.081Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 304.817217] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e4ed2d7ec330e34970fe7b6","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e4ed2d7ec330e34970fe7b6: start","job_status":"start","queue_duration_s":1.480858,"scheduling_latency_s":1.480874}1542gitlab # [ 304.829408] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e4ed2d7ec330e34970fe7b6","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e4ed2d7ec330e34970fe7b6: done: 0.013832 sec","job_status":"done","queue_duration_s":1.480858,"scheduling_latency_s":1.480874,"redis_calls":13,"redis_duration_s":0.001361,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013832,"completed_at":"2026-09-08T18:41:04.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 304.867437] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2a9b4951c6093d7a243ce91","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2a9b4951c6093d7a243ce91: start","job_status":"start","queue_duration_s":1.495709,"scheduling_latency_s":1.495726}1544gitlab # [ 304.879552] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2a9b4951c6093d7a243ce91","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2a9b4951c6093d7a243ce91: done: 0.013901 sec","job_status":"done","queue_duration_s":1.495709,"scheduling_latency_s":1.495726,"redis_calls":13,"redis_duration_s":0.0013779999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013901,"completed_at":"2026-09-08T18:41:04.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 304.917724] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c37fb47163a20a6bf01bce0e","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c37fb47163a20a6bf01bce0e: start","job_status":"start","queue_duration_s":1.510678,"scheduling_latency_s":1.510696}1546gitlab # [ 304.929712] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c37fb47163a20a6bf01bce0e","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c37fb47163a20a6bf01bce0e: done: 0.014024 sec","job_status":"done","queue_duration_s":1.510678,"scheduling_latency_s":1.510696,"redis_calls":13,"redis_duration_s":0.00142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014024,"completed_at":"2026-09-08T18:41:04.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 304.968810] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84757c6b78d8136ecbcfbb01","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84757c6b78d8136ecbcfbb01: start","job_status":"start","queue_duration_s":1.525774,"scheduling_latency_s":1.525791}1548gitlab # [ 304.981378] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84757c6b78d8136ecbcfbb01","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84757c6b78d8136ecbcfbb01: done: 0.014047 sec","job_status":"done","queue_duration_s":1.525774,"scheduling_latency_s":1.525791,"redis_calls":13,"redis_duration_s":0.001353,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000856,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014047,"completed_at":"2026-09-08T18:41:04.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 305.019343] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35cdeca7484a9b907884c34a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35cdeca7484a9b907884c34a: start","job_status":"start","queue_duration_s":1.540888,"scheduling_latency_s":1.540905}1550gitlab # [ 305.031538] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35cdeca7484a9b907884c34a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35cdeca7484a9b907884c34a: done: 0.013803 sec","job_status":"done","queue_duration_s":1.540888,"scheduling_latency_s":1.540905,"redis_calls":13,"redis_duration_s":0.001356,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013803,"completed_at":"2026-09-08T18:41:04.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 305.069709] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ac173478498fc33e07a5064","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ac173478498fc33e07a5064: start","job_status":"start","queue_duration_s":1.555746,"scheduling_latency_s":1.555764}1552gitlab # [ 305.081871] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ac173478498fc33e07a5064","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ac173478498fc33e07a5064: done: 0.01408 sec","job_status":"done","queue_duration_s":1.555746,"scheduling_latency_s":1.555764,"redis_calls":13,"redis_duration_s":0.00139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01408,"completed_at":"2026-09-08T18:41:04.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 305.119758] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3de5a6608cfbd5c06837e17a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3de5a6608cfbd5c06837e17a: start","job_status":"start","queue_duration_s":1.570869,"scheduling_latency_s":1.570886}1554gitlab # [ 305.131901] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3de5a6608cfbd5c06837e17a","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3de5a6608cfbd5c06837e17a: done: 0.014295 sec","job_status":"done","queue_duration_s":1.570869,"scheduling_latency_s":1.570886,"redis_calls":13,"redis_duration_s":0.001363,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014295,"completed_at":"2026-09-08T18:41:04.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 305.171739] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200f87f002ae856528c4b522","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200f87f002ae856528c4b522: start","job_status":"start","queue_duration_s":1.586334,"scheduling_latency_s":1.586352}1556gitlab # [ 305.184386] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200f87f002ae856528c4b522","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200f87f002ae856528c4b522: done: 0.014958 sec","job_status":"done","queue_duration_s":1.586334,"scheduling_latency_s":1.586352,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014958,"completed_at":"2026-09-08T18:41:04.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 305.222948] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8740f852d2d5d7913bbcfe7b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8740f852d2d5d7913bbcfe7b: start","job_status":"start","queue_duration_s":1.602539,"scheduling_latency_s":1.60256}1558gitlab # [ 305.235305] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8740f852d2d5d7913bbcfe7b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8740f852d2d5d7913bbcfe7b: done: 0.014572 sec","job_status":"done","queue_duration_s":1.602539,"scheduling_latency_s":1.60256,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014572,"completed_at":"2026-09-08T18:41:04.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559gitlab # [ 305.273429] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7822753bab7cf252ff47532","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7822753bab7cf252ff47532: start","job_status":"start","queue_duration_s":1.618152,"scheduling_latency_s":1.618169}1560gitlab # [ 305.285641] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7822753bab7cf252ff47532","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7822753bab7cf252ff47532: done: 0.013938 sec","job_status":"done","queue_duration_s":1.618152,"scheduling_latency_s":1.618169,"redis_calls":13,"redis_duration_s":0.001339,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013938,"completed_at":"2026-09-08T18:41:04.233Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 305.324268] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f75d0aed1da5426a5b5932eb","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f75d0aed1da5426a5b5932eb: start","job_status":"start","queue_duration_s":1.633179,"scheduling_latency_s":1.633198}1562gitlab # [ 305.336537] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f75d0aed1da5426a5b5932eb","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f75d0aed1da5426a5b5932eb: done: 0.01456 sec","job_status":"done","queue_duration_s":1.633179,"scheduling_latency_s":1.633198,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01456,"completed_at":"2026-09-08T18:41:04.248Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 305.376092] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76db9fad8caea8c3072c426d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76db9fad8caea8c3072c426d: start","job_status":"start","queue_duration_s":1.648765,"scheduling_latency_s":1.648782}1564gitlab # [ 305.388347] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76db9fad8caea8c3072c426d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76db9fad8caea8c3072c426d: done: 0.013803 sec","job_status":"done","queue_duration_s":1.648765,"scheduling_latency_s":1.648782,"redis_calls":13,"redis_duration_s":0.001356,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013803,"completed_at":"2026-09-08T18:41:04.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565gitlab # [ 305.426644] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f93a417b3a2b88fe8acba44b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f93a417b3a2b88fe8acba44b: start","job_status":"start","queue_duration_s":1.663605,"scheduling_latency_s":1.663621}1566gitlab # [ 305.439249] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f93a417b3a2b88fe8acba44b","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f93a417b3a2b88fe8acba44b: done: 0.013836 sec","job_status":"done","queue_duration_s":1.663605,"scheduling_latency_s":1.663621,"redis_calls":13,"redis_duration_s":0.00135,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013836,"completed_at":"2026-09-08T18:41:04.278Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 305.477500] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"261da78f8c6be45014a45e5d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-261da78f8c6be45014a45e5d: start","job_status":"start","queue_duration_s":1.678474,"scheduling_latency_s":1.678491}1568gitlab # [ 305.489809] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"261da78f8c6be45014a45e5d","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-261da78f8c6be45014a45e5d: done: 0.014031 sec","job_status":"done","queue_duration_s":1.678474,"scheduling_latency_s":1.678491,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014031,"completed_at":"2026-09-08T18:41:04.293Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569gitlab # [ 305.528052] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5609a14625cf8a3bb647b509","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5609a14625cf8a3bb647b509: start","job_status":"start","queue_duration_s":1.693589,"scheduling_latency_s":1.693606}1570gitlab # [ 305.540119] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5609a14625cf8a3bb647b509","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5609a14625cf8a3bb647b509: done: 0.013868 sec","job_status":"done","queue_duration_s":1.693589,"scheduling_latency_s":1.693606,"redis_calls":13,"redis_duration_s":0.0013410000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013868,"completed_at":"2026-09-08T18:41:04.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571gitlab # [ 305.579290] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4388875673d7efd93df16af3","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4388875673d7efd93df16af3: start","job_status":"start","queue_duration_s":1.708484,"scheduling_latency_s":1.708502}1572gitlab # [ 305.591350] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4388875673d7efd93df16af3","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4388875673d7efd93df16af3: done: 0.013868 sec","job_status":"done","queue_duration_s":1.708484,"scheduling_latency_s":1.708502,"redis_calls":13,"redis_duration_s":0.001412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013868,"completed_at":"2026-09-08T18:41:04.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573gitlab # [ 305.629659] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d2e118ee9c4b5cf04d31627","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d2e118ee9c4b5cf04d31627: start","job_status":"start","queue_duration_s":1.723404,"scheduling_latency_s":1.723422}1574gitlab # [ 305.641883] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d2e118ee9c4b5cf04d31627","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d2e118ee9c4b5cf04d31627: done: 0.014023 sec","job_status":"done","queue_duration_s":1.723404,"scheduling_latency_s":1.723422,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014023,"completed_at":"2026-09-08T18:41:04.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575gitlab # [ 305.680069] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fda8f30c9a16b3d93529e97f","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fda8f30c9a16b3d93529e97f: start","job_status":"start","queue_duration_s":1.738489,"scheduling_latency_s":1.738507}1576gitlab # [ 305.692175] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:04.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fda8f30c9a16b3d93529e97f","created_at":"2026-09-08T18:41:02.519Z","trace_propagation_headers":{"sentry-trace":"c4b9930e3cc3492587f849c04166b963-e6d31632c9e847a4","baggage":"sentry-trace_id=c4b9930e3cc3492587f849c04166b963,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0e24cf40f67b90f4c0e8f1a46b488181","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:02.601Z","job_size_bytes":2,"pid":2035,"sidekiq_tid":"maor","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fda8f30c9a16b3d93529e97f: done: 0.014361 sec","job_status":"done","queue_duration_s":1.738489,"scheduling_latency_s":1.738507,"redis_calls":13,"redis_duration_s":0.001388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014361,"completed_at":"2026-09-08T18:41:04.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577gitlab # [ 309.316622] redis[791]: 10 changes in 300 seconds. Saving...1578gitlab # [ 309.322244] redis[791]: Background saving started by pid 32931579gitlab # [ 309.327841] redis[3293]: BGSAVE done, 498 keys saved, 0 keys skipped, 125850 bytes written.1580gitlab # [ 309.332493] redis[3293]: DB saved on disk1581gitlab # [ 309.334720] redis[3293]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1582gitlab # [ 309.417732] redis[791]: Background saving terminated with success1583gitlab # [ 310.567707] postgres[864]: [864] LOG: checkpoint starting: time1584gitlab # [ 312.322174] postgres[3283]: [3283] ERROR: relation "table_without_models" does not exist1585gitlab # [ 312.322514] postgres[3283]: [3283] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1586gitlab # [ 312.447329] gitaly[906]: time="2026-09-08T18:41:13.215Z" level=info msg="burden monitor stats" pid=9061587gitlab # [ 317.958263] runuser[2948]: pam_unix(runuser:session): session closed for user gitlab1588gitlab # [ 317.959734] setup-gitlab-start[2921]: + systemctl restart gitlab-sidekiq.service1589gitlab # [ 317.996903] systemd[1]: Stopping gitlab-sidekiq.service...1590gitlab # [ 318.002212] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:18.770Z","message":"Shutting down"}1591gitlab # [ 318.006477] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:18.771Z","message":"Scheduler exiting..."}1592gitlab # [ 318.010763] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:18.772Z","message":"Terminating quiet threads for default capsule"}1593gitlab # [ 318.015951] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:18.772Z","message":"Scheduler exiting..."}1594gitlab # [ 318.504145] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:19.273Z","message":"Pausing to allow jobs to finish..."}1595gitlab # [ 319.007975] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:19.776Z","message":"Bye!"}1596gitlab # [ 319.048713] sidekiq-cluster[2035]: {"severity":"INFO","time":"2026-09-08T18:41:19.818Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2035,"worker_id":"sidekiq_0","memwd_rss_bytes":954974208,"retry":0}1597gitlab # [ 319.798611] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1598gitlab # [ 319.799271] systemd[1]: Stopped gitlab-sidekiq.service.1599gitlab # [ 319.804175] systemd[1]: gitlab-sidekiq.service: Consumed 1min 49.644s CPU time over 3min 15.469s wall clock time, 947.5M memory peak, 4.5M read from disk.1600gitlab # [ 319.810211] systemd[1]: Started gitlab-sidekiq.service.1601gitlab # [ 319.813695] setup-gitlab-start[2921]: + mkdir -p /var/lib/secrets1602gitlab # [ 319.816239] setup-gitlab-start[2921]: + echo nixbot-test-token-of-20-chars1603gitlab # [ 319.819030] setup-gitlab-start[2921]: + chmod 644 /var/lib/secrets/gitlab-token1604gitlab # [ 319.822122] setup-gitlab-start[2921]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1605gitlab # [ 320.252375] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:21 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 3871606gitlab # [ 322.791800] gitaly[906]: time="2026-09-08T18:41:23.560Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:33.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:23.560" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 322.792415] gitaly[906]: time="2026-09-08T18:41:23.561Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:33.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:23.560" grpc.time_ms=0.784 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1608gitlab # [ 323.136592] gitaly[906]: time="2026-09-08T18:41:23.906Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:33.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:23.906" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1609gitlab # [ 323.169032] gitaly[906]: time="2026-09-08T18:41:23.906Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:33.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:23.906" grpc.time_ms=0.279 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 323.243601] gitaly[906]: time="2026-09-08T18:41:24.013Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW diskcache=3dd3795f-d6d9-4dff-bceb-1fdcfe610140 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T18:41:33.063" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:23.964" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1611gitlab # [ 323.274438] gitaly[906]: time="2026-09-08T18:41:24.014Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=25 command.spawn_token_fork_ms=4 command.system_time_ms=13 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T18:41:33.063" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:23.964" grpc.time_ms=49.333 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1612gitlab # [ 323.303271] gitaly[906]: time="2026-09-08T18:41:24.060Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:34.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.059" pid=906 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1613gitlab # [ 323.319626] gitaly[906]: time="2026-09-08T18:41:24.061Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:34.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.059" grpc.time_ms=1.403 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 323.334082] gitaly[906]: time="2026-09-08T18:41:24.097Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:41:34.032" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.083" grpc.time_ms=14.133 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615gitlab # [ 323.554201] gitaly[906]: time="2026-09-08T18:41:24.324Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:34.273" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.324" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1616gitlab # [ 323.585600] gitaly[906]: time="2026-09-08T18:41:24.324Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:34.273" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.324" grpc.time_ms=0.33 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1617gitlab # [ 323.629667] gitaly[906]: time="2026-09-08T18:41:24.399Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW diskcache=a884f5c8-b578-42a9-b928-7e75f695a913 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T18:41:34.000" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.361" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1618gitlab # [ 323.657772] gitaly[906]: time="2026-09-08T18:41:24.400Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1441 command.oublock=32 command.real_time_ms=19 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T18:41:34.000" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.361" grpc.time_ms=38.408 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1619gitlab # [ 323.687234] gitaly[906]: time="2026-09-08T18:41:24.421Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:34.060" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.421" pid=906 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1620gitlab # [ 323.704357] gitaly[906]: time="2026-09-08T18:41:24.422Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:34.060" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.421" grpc.time_ms=0.834 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1621gitlab # [ 323.719117] gitaly[906]: time="2026-09-08T18:41:24.452Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:41:34.080" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:24.441" grpc.time_ms=10.551 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1622gitlab # [ 324.503147] gitaly[906]: time="2026-09-08T18:41:25.272Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:41:35.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:25.271" grpc.time_ms=0.191 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1623gitlab # [ 324.536198] gitaly[906]: time="2026-09-08T18:41:25.280Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T18:41:35.230" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:25.280" grpc.time_ms=0.242 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1624gitlab # [ 324.870272] sidekiq-cluster[3339]: {"severity":"INFO","time":"2026-09-08T18:41:25.608Z","message":"Starting cluster with 1 processes"}1625gitlab # [ 325.210845] gitaly[906]: time="2026-09-08T18:41:25.980Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M215684E54628ZQSJ28Q3QQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:41:35.062" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:25.963" grpc.time_ms=17.138 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1626gitlab # [ 325.604437] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:26 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 53351627gitlab # [ 325.608636] setup-gitlab-start[3352]: {"id":1,"message":null,"error":null}1628gitlab # [ 330.910790] gitaly[906]: time="2026-09-08T18:41:31.680Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2156J98DMY4NHPN96A3SSAC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:41:41.106" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:31.657" grpc.time_ms=22.193 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1629gitlab # [ 331.095013] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:31 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 4301630gitlab # [ 331.102067] setup-gitlab-start[2921]: + mkdir -p /tmp/test-flake1631gitlab # [ 331.104775] setup-gitlab-start[2921]: + cd /tmp/test-flake1632gitlab # [ 331.108192] setup-gitlab-start[2921]: + git init -b master1633gitlab # [ 331.133546] setup-gitlab-start[3439]: Initialized empty Git repository in /tmp/test-flake/.git/1634gitlab # [ 331.137304] setup-gitlab-start[2921]: + git config user.name test1635gitlab # [ 331.142116] setup-gitlab-start[2921]: + git config user.email test@example.com1636gitlab # [ 331.146583] setup-gitlab-start[2921]: + cat1637gitlab # [ 331.150265] setup-gitlab-start[2921]: + git add flake.nix1638gitlab # [ 331.155478] setup-gitlab-start[2921]: + git commit -m 'initial commit'1639gitlab # [ 331.175445] setup-gitlab-start[3445]: [master (root-commit) da5a12e] initial commit1640gitlab # [ 331.178956] setup-gitlab-start[3445]: 1 file changed, 10 insertions(+)1641gitlab # [ 331.181812] setup-gitlab-start[3445]: create mode 100644 flake.nix1642gitlab # [ 331.184485] setup-gitlab-start[2921]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1643gitlab # [ 331.189344] setup-gitlab-start[2921]: + git push -u origin master1644gitlab # [ 331.288866] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 541645gitlab # [ 331.433597] gitaly[906]: time="2026-09-08T18:41:32.203Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2156JWVAHGAETKB4EBME8PH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:32.184" grpc.time_ms=18.643 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1646gitlab # [ 331.472590] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1451647gitlab # [ 332.148345] gitaly[906]: time="2026-09-08T18:41:32.917Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:41:42.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:32.899" grpc.time_ms=17.497 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1648gitlab # [ 332.275732] gitaly[906]: time="2026-09-08T18:41:33.045Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=6 command.system_time_ms=9 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-08T18:42:03.024" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-08T18:41:33.025" grpc.time_ms=20.162 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1649gitlab # [ 332.334696] gitaly[906]: time="2026-09-08T18:41:33.104Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 duration_ms=329ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/allowed"1650gitlab # [ 332.390209] gitaly[906]: time="2026-09-08T18:41:33.160Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 duration_ms=54ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/pre_receive"1651gitlab # [ 332.403821] gitaly[906]: time="2026-09-08T18:41:33.160Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:41:32.772" grpc.time_ms=387.842 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1652gitlab # [ 332.475871] gitaly[906]: time="2026-09-08T18:41:33.245Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:41:33.244" grpc.time_ms=1.444 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1653gitlab # [ 332.544373] gitaly[906]: time="2026-09-08T18:41:33.314Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:41:33.313" grpc.time_ms=1.065 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1654gitlab # [ 332.590853] gitaly[906]: time="2026-09-08T18:41:33.360Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:41:33.359" grpc.time_ms=0.867 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1655gitlab # [ 332.740686] gitaly[906]: time="2026-09-08T18:41:33.510Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:41:43.134" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:33.495" grpc.time_ms=15.039 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1656gitlab # [ 332.835996] gitaly[906]: time="2026-09-08T18:41:33.605Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=30 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=5 command.system_time_ms=30 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-08T18:41:43.205" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:33.565" grpc.time_ms=39.403 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1657gitlab # [ 332.876558] gitaly[906]: time="2026-09-08T18:41:33.624Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 duration_ms=212ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/post_receive"1658gitlab # [ 332.884561] gitaly[906]: time="2026-09-08T18:41:33.625Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:41:33.410" grpc.time_ms=214.967 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1659gitlab # [ 332.908279] gitaly[906]: time="2026-09-08T18:41:33.638Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 diskcache=c1ac563b-8048-42a7-97de-46fcc0f29d47 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:32.668" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1660gitlab # [ 332.925739] setup-gitlab-start[3449]: To http://gitlab/root/test-flake.git1661gitlab # [ 332.927109] setup-gitlab-start[3449]: * [new branch] master -> master1662gitlab # [ 332.928493] setup-gitlab-start[3449]: branch 'master' set up to track 'origin/master'.1663gitlab # [ 332.930482] gitlab-workhorse[823]: time="2026-09-08T18:41:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521664gitlab # [ 332.933195] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:33 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 14131665gitlab # [ 332.936231] systemd[1]: Finished setup-gitlab.service.1666gitlab # [ 332.937297] gitaly[906]: time="2026-09-08T18:41:33.638Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=350 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13822 command.oublock=32 command.real_time_ms=962 command.spawn_token_fork_ms=0 command.system_time_ms=200 command.user_time_ms=150 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:32.668" grpc.time_ms=969.85 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1667gitlab # [ 332.955239] systemd[1]: Starting nixbot CI service...1668gitlab: (finished: waiting for unit setup-gitlab.service, in 333.99 seconds)1669gitlab: waiting for unit nixbot.service1670gitlab # [ 334.357262] nixbot[3531]: {"ts": "2026-09-08T18:41:35.126Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1671gitlab # [ 334.384135] nixbot[3531]: {"ts": "2026-09-08T18:41:35.153Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1672gitlab # [ 334.401605] nixbot[3531]: {"ts": "2026-09-08T18:41:35.171Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1673gitlab # [ 334.408303] nixbot[3531]: {"ts": "2026-09-08T18:41:35.176Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1674gitlab # [ 334.414907] nixbot[3531]: {"ts": "2026-09-08T18:41:35.181Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1675gitlab # [ 334.421276] nixbot[3531]: {"ts": "2026-09-08T18:41:35.187Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1676gitlab # [ 334.427886] nixbot[3531]: {"ts": "2026-09-08T18:41:35.194Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1677gitlab # [ 334.434299] nixbot[3531]: {"ts": "2026-09-08T18:41:35.197Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1678gitlab # [ 334.440880] nixbot[3531]: {"ts": "2026-09-08T18:41:35.208Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1679gitlab # [ 334.447258] nixbot[3531]: {"ts": "2026-09-08T18:41:35.210Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1680gitlab # [ 334.455177] nixbot[3531]: {"ts": "2026-09-08T18:41:35.219Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1681gitlab # [ 334.461297] nixbot[3531]: {"ts": "2026-09-08T18:41:35.230Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1682gitlab # [ 334.467213] nixbot[3531]: {"ts": "2026-09-08T18:41:35.235Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1683gitlab # [ 334.473482] nixbot[3531]: {"ts": "2026-09-08T18:41:35.238Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1684gitlab # [ 334.479820] nixbot[3531]: {"ts": "2026-09-08T18:41:35.244Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1685gitlab # [ 334.486038] nixbot[3531]: {"ts": "2026-09-08T18:41:35.246Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1686gitlab # [ 334.491457] nixbot[3531]: {"ts": "2026-09-08T18:41:35.249Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1687gitlab # [ 334.497127] nixbot[3531]: {"ts": "2026-09-08T18:41:35.252Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1688gitlab # [ 334.502270] nixbot[3531]: {"ts": "2026-09-08T18:41:35.259Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1689gitlab # [ 334.507272] nixbot[3531]: {"ts": "2026-09-08T18:41:35.262Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1690gitlab # [ 334.512603] nixbot[3531]: {"ts": "2026-09-08T18:41:35.265Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1691gitlab # [ 334.517584] nixbot[3531]: {"ts": "2026-09-08T18:41:35.272Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1692gitlab # [ 334.522457] nixbot[3531]: {"ts": "2026-09-08T18:41:35.275Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1693gitlab # [ 334.527231] nixbot[3531]: {"ts": "2026-09-08T18:41:35.278Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1694gitlab # [ 334.531645] nixbot[3531]: {"ts": "2026-09-08T18:41:35.281Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1695gitlab # [ 334.535889] nixbot[3531]: {"ts": "2026-09-08T18:41:35.286Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1696gitlab # [ 334.539986] nixbot[3531]: {"ts": "2026-09-08T18:41:35.289Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1697gitlab # [ 334.543677] nixbot[3531]: {"ts": "2026-09-08T18:41:35.291Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1698gitlab # [ 334.547214] nixbot[3531]: {"ts": "2026-09-08T18:41:35.293Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1699gitlab # [ 334.550766] nixbot[3531]: {"ts": "2026-09-08T18:41:35.317Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1700gitlab # [ 334.555070] nixbot[3531]: {"ts": "2026-09-08T18:41:35.324Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1701gitlab # [ 334.558920] nixbot[3531]: {"ts": "2026-09-08T18:41:35.328Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1702gitlab # [ 334.562700] nixbot[3531]: {"ts": "2026-09-08T18:41:35.331Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1703gitlab # [ 334.569605] nixbot[3531]: {"ts": "2026-09-08T18:41:35.339Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1704gitlab # [ 335.044219] nixbot[3531]: INFO: Started server process [3531]1705gitlab # [ 335.046943] nixbot[3531]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1706gitlab # [ 335.072245] nixbot[3531]: INFO: ::ffff:127.0.0.1:60782 - "GET /health HTTP/1.1" 200 OK1707gitlab # [ 335.082802] systemd[1]: Started nixbot CI service.1708gitlab # [ 335.086099] systemd[1]: Reached target Multi-User System.1709gitlab # [ 335.089311] systemd[1]: Startup finished in 905ms (kernel) + 4.422s (initrd) + 5min 29.761s (userspace) = 5min 35.088s.1710gitlab # [ 335.565733] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:36 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 4921711gitlab # [ 335.573968] nixbot[3531]: {"ts": "2026-09-08T18:41:36.336Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1712gitlab # [ 335.604548] nixbot[3531]: {"ts": "2026-09-08T18:41:36.374Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1713gitlab: (finished: waiting for unit nixbot.service, in 2.14 seconds)1714gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1715gitlab # [ 335.719902] nixbot[3531]: INFO: ::ffff:127.0.0.1:60798 - "GET /health HTTP/1.1" 200 OK1716gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds)1717subtest: project discovered and webhook registered1718gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1719gitlab # [ 335.779340] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1661720gitlab # [ 335.792704] nixbot[3531]: {"ts": "2026-09-08T18:41:36.557Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1721gitlab # [ 335.805295] nixbot[3531]: {"ts": "2026-09-08T18:41:36.561Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1722gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1723gitlab # [ 335.896958] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 1321724gitlab # [ 336.036830] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 2321725gitlab # [ 336.043506] nixbot[3531]: {"ts": "2026-09-08T18:41:36.808Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1726gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1727gitlab # [ 337.077253] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 1181728gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1729(finished: subtest: project discovered and webhook registered, in 1.36 seconds)1730subtest: push delivers the webhook1731gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1732gitlab # [ 337.212166] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:37 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 11611733gitlab # [ 337.226794] nixbot[3531]: {"ts": "2026-09-08T18:41:37.991Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1734gitlab # [ 337.344223] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 1091735gitlab # [ 337.350783] nixbot[3531]: {"ts": "2026-09-08T18:41:38.115Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1736gitlab # [ 337.436330] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:38 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 861737gitlab # [ 337.443762] nixbot[3531]: {"ts": "2026-09-08T18:41:38.207Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1738gitlab # [ 337.529667] gitaly[906]: time="2026-09-08T18:41:38.297Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2156RX5ZX7DKXSR0FY7B1C4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-08T18:42:08.269" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:38.270" grpc.time_ms=26.094 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739gitlab # [ 337.556134] gitaly[906]: time="2026-09-08T18:41:38.326Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156RX5ZX7DKXSR0FY7B1C4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:42:08.323" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:41:38.324" grpc.time_ms=1.328 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1740gitlab # [ 337.587551] gitaly[906]: time="2026-09-08T18:41:38.357Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156RX5ZX7DKXSR0FY7B1C4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-08T18:41:48.297" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:41:38.348" grpc.time_ms=9.043 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741gitlab # [ 337.631842] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1881742gitlab # [ 337.639277] nixbot[3531]: {"ts": "2026-09-08T18:41:38.402Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1743gitlab # [ 337.723530] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 871744gitlab # [ 337.731224] nixbot[3531]: {"ts": "2026-09-08T18:41:38.494Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1745gitlab # [ 337.739449] nixbot[3531]: {"ts": "2026-09-08T18:41:38.499Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1746gitlab # [ 337.855578] gitaly[906]: time="2026-09-08T18:41:38.625Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156S7TJC3P983R0W31MPQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:38.611" grpc.time_ms=13.779 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1747gitlab # [ 337.894130] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 711748gitlab # [ 337.974663] gitaly[906]: time="2026-09-08T18:41:38.744Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2156SA4HN4V03CBVVWVJW4S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:38.725" pid=906 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1749gitlab # [ 338.005230] gitaly[906]: time="2026-09-08T18:41:38.744Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2156SA4HN4V03CBVVWVJW4S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:38.725" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1750gitlab # [ 338.032535] gitlab-workhorse[823]: time="2026-09-08T18:41:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131751gitlab # [ 338.035585] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1181752gitlab # [ 338.038745] gitaly[906]: time="2026-09-08T18:41:38.745Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2156SA4HN4V03CBVVWVJW4S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:38.725" grpc.time_ms=19.702 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1753gitlab # [ 338.108548] gitaly[906]: time="2026-09-08T18:41:38.878Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2156SE11E26YE741PXQ077Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:38.846" pid=906 remote_ip="2001:db8:1::1" request_sha=827406d1e8882b73d2508414abe5f2ad7c0e0e6b response_bytes=223 span.kind=server system=grpc1754gitlab # [ 338.133446] gitaly[906]: time="2026-09-08T18:41:38.878Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=2 command.system_time_ms=20 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2156SE11E26YE741PXQ077Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:38.846" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1755gitlab # [ 338.162024] gitlab-workhorse[823]: time="2026-09-08T18:41:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231756gitlab # [ 338.165248] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1261757gitlab # [ 338.169170] gitaly[906]: time="2026-09-08T18:41:38.879Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=2 command.system_time_ms=20 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2156SE11E26YE741PXQ077Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:38.846" grpc.time_ms=32.997 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1758gitlab # [ 338.275395] gitaly[906]: time="2026-09-08T18:41:39.045Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156SMHQTASER3JVJPEAGQR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.027" grpc.time_ms=17.564 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1759gitlab # [ 338.292212] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 841760gitlab # [ 338.407138] gitaly[906]: time="2026-09-08T18:41:39.177Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2156SQ9J612A899J6GHEPM3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.158" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1761gitlab # [ 338.435595] gitaly[906]: time="2026-09-08T18:41:39.177Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156SQ9J612A899J6GHEPM3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.158" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1762gitlab # [ 338.460579] gitlab-workhorse[823]: time="2026-09-08T18:41:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671763gitlab # [ 338.463439] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1291764gitlab # [ 338.466509] gitaly[906]: time="2026-09-08T18:41:39.177Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156SQ9J612A899J6GHEPM3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.158" grpc.time_ms=19.251 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1765gitlab # [ 338.609792] gitaly[906]: time="2026-09-08T18:41:39.378Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156SXSTNSY52WVG19J8HM9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.359" grpc.time_ms=19.403 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1766gitlab # [ 338.647126] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1221767gitlab # [ 338.749676] gitaly[906]: time="2026-09-08T18:41:39.519Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2156T1QDCT0PWGKMNPC4N9W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.486" pid=906 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1768gitlab # [ 338.781674] gitaly[906]: time="2026-09-08T18:41:39.519Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=1 command.system_time_ms=21 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T1QDCT0PWGKMNPC4N9W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.486" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1769gitlab # [ 338.812880] gitlab-workhorse[823]: time="2026-09-08T18:41:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091770gitlab # [ 338.816269] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1371771gitlab # [ 338.819701] gitaly[906]: time="2026-09-08T18:41:39.520Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=1 command.system_time_ms=21 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T1QDCT0PWGKMNPC4N9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:39.486" grpc.time_ms=33.796 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1772gitlab # [ 338.941308] gitaly[906]: time="2026-09-08T18:41:39.711Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:42:09.009" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:41:39.710" grpc.time_ms=0.947 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 339.034852] gitaly[906]: time="2026-09-08T18:41:39.804Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-08T18:42:09.084" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:39.785" grpc.time_ms=18.99 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774gitlab # [ 339.084287] gitaly[906]: time="2026-09-08T18:41:39.854Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:41:49.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:39.838" grpc.time_ms=15.808 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 339.128714] gitaly[906]: time="2026-09-08T18:41:39.898Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:41:49.051" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:39.881" grpc.time_ms=16.863 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1776gitlab # [ 339.269730] gitaly[906]: time="2026-09-08T18:41:40.039Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T18:41:50.008" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:40.038" grpc.time_ms=0.598 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1777gitlab # [ 339.303506] gitaly[906]: time="2026-09-08T18:41:40.063Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T18:41:50.018" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:40.049" grpc.time_ms=13.655 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1778gitlab # [ 339.354817] gitaly[906]: time="2026-09-08T18:41:40.124Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:41:40.123" grpc.time_ms=0.705 pid=906 span.kind=server system=grpc1779gitlab # [ 339.404960] gitaly[906]: time="2026-09-08T18:41:40.174Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:41:40.173" grpc.time_ms=0.96 pid=906 span.kind=server system=grpc1780gitlab # [ 339.432531] gitaly[906]: time="2026-09-08T18:41:40.192Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN diskcache=8fa9715b-b252-4474-9c4c-9b8e77ed17b0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T18:41:50.035" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:40.066" pid=906 ref=refs/keep-around/da5a12ee3cffbf5b7dee97e408207083a88bbf1e remote_ip="2001:db8:1::1" revision=da5a12ee3cffbf5b7dee97e408207083a88bbf1e span.kind=server system=grpc username=root1781gitlab # [ 339.456553] gitaly[906]: time="2026-09-08T18:41:40.192Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=118 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5254 command.oublock=8 command.real_time_ms=119 command.spawn_token_fork_ms=3 command.system_time_ms=60 command.user_time_ms=58 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T18:41:50.035" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:41:40.066" grpc.time_ms=126.015 pid=906 ref=refs/keep-around/da5a12ee3cffbf5b7dee97e408207083a88bbf1e remote_ip="2001:db8:1::1" revision=da5a12ee3cffbf5b7dee97e408207083a88bbf1e span.kind=server system=grpc username=root1782gitlab # [ 339.478987] gitaly[906]: time="2026-09-08T18:41:40.199Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:42:10.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:41:40.198" grpc.time_ms=0.764 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783gitlab # [ 339.720289] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 9001784gitlab # [ 339.729228] nixbot[3531]: {"ts": "2026-09-08T18:41:40.490Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e \"HTTP/1.1 201 Created\""}1785gitlab # [ 339.738100] nixbot[3531]: {"ts": "2026-09-08T18:41:40.505Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-da5a12ee3cff-0"}1786gitlab # [ 340.042117] nixbot[3531]: {"ts": "2026-09-08T18:41:40.811Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-da5a12ee3cff-0", "argv0": "bwrap"}1787gitlab # [ 340.212963] systemd[1]: Started Nix Daemon.1788gitlab # [ 340.331831] nix-daemon[3709]: accepted connection from pid 3704, user nixbot1789gitlab # [ 345.001430] nix-daemon[3709]: accepted connection from pid 3758, user nixbot1790gitlab # [ 345.025394] nix-daemon[3709]: accepted connection from pid 3704, user nixbot1791gitlab # [ 349.681376] nix-daemon[3709]: accepted connection from pid 3704, user nixbot1792gitlab # [ 349.873118] gitaly[906]: time="2026-09-08T18:41:50.642Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M21574Y9X4ZTVP1A37CC1RBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:42:20.531" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:41:50.632" grpc.time_ms=9.483 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793gitlab # [ 349.947160] gitaly[906]: time="2026-09-08T18:41:50.716Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21574Y9X4ZTVP1A37CC1RBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:42:00.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:50.698" grpc.time_ms=17.868 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1794gitlab # [ 350.046922] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2781795gitlab # [ 350.055820] nixbot[3531]: {"ts": "2026-09-08T18:41:50.817Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e \"HTTP/1.1 201 Created\""}1796gitlab # [ 350.129172] gitaly[906]: time="2026-09-08T18:41:50.899Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2157577N2P29K2326FCS0JS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:42:20.196" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:41:50.897" grpc.time_ms=1.081 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797gitlab # [ 350.181491] gitaly[906]: time="2026-09-08T18:41:50.951Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2157577N2P29K2326FCS0JS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:42:00.103" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:50.934" grpc.time_ms=16.844 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab # [ 350.242857] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1881799gitlab # [ 350.250806] nixbot[3531]: {"ts": "2026-09-08T18:41:51.013Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e \"HTTP/1.1 201 Created\""}1800gitlab # [ 350.315288] gitaly[906]: time="2026-09-08T18:41:51.085Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21575DKW3JBK8CQ2865WWM3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:42:21.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:41:51.084" grpc.time_ms=0.868 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 350.391733] gitaly[906]: time="2026-09-08T18:41:51.161Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21575DKW3JBK8CQ2865WWM3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:42:01.113" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:41:51.143" grpc.time_ms=17.944 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802gitlab # [ 350.548246] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2851803gitlab # [ 350.559958] nixbot[3531]: {"ts": "2026-09-08T18:41:51.315Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e \"HTTP/1.1 201 Created\""}1804gitlab # [ 351.215898] gitaly[906]: time="2026-09-08T18:41:51.984Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M215767SBCWXPX7HAVF70JM6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:51.970" grpc.time_ms=13.978 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1805gitlab # [ 351.255165] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1211806gitlab # [ 351.381969] gitaly[906]: time="2026-09-08T18:41:52.151Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M21576BPNTJ552PEZC1XECDD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:52.132" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1807gitlab # [ 351.415832] gitaly[906]: time="2026-09-08T18:41:52.152Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21576BPNTJ552PEZC1XECDD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:52.132" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1808gitlab # [ 351.439838] gitlab-workhorse[823]: time="2026-09-08T18:41:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671809gitlab # [ 351.442842] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1621810gitlab # [ 351.445775] gitaly[906]: time="2026-09-08T18:41:52.153Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21576BPNTJ552PEZC1XECDD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:52.132" grpc.time_ms=20.483 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1811gitlab # [ 351.590630] gitaly[906]: time="2026-09-08T18:41:52.360Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21576JZEXD4PSW8MCC10Z94 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:52.343" grpc.time_ms=16.752 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1812gitlab # [ 351.623256] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1381813gitlab # [ 351.717597] gitaly[906]: time="2026-09-08T18:41:52.487Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M21576QCF1V9ZSWX78RVYGGV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:52.468" pid=906 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1814gitlab # [ 351.735754] gitaly[906]: time="2026-09-08T18:41:52.487Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21576QCF1V9ZSWX78RVYGGV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:52.468" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1815gitlab # [ 351.754982] gitlab-workhorse[823]: time="2026-09-08T18:41:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671816gitlab # [ 351.758618] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:41:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1241817gitlab # [ 351.761135] gitaly[906]: time="2026-09-08T18:41:52.488Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21576QCF1V9ZSWX78RVYGGV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:41:52.468" grpc.time_ms=19.486 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1818gitlab # [ 365.708670] sidekiq-cluster[3399]: 2026-09-08T18:42:06.477Z pid=3399 tid=yr3 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3399, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426344448}1819gitlab # [ 366.010018] sidekiq-cluster[3399]: W, [2026-09-08T18:42:06.780032 #3399] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1820gitlab # [ 366.147964] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:06.917Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1821gitlab # [ 369.129308] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:09.898Z","message":"GitLab reliable fetch activated!"}1822gitlab # [ 369.129628] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:09.898Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1823gitlab # [ 405.039649] postgres[4246]: [4246] ERROR: relation "table_without_models" does not exist1824gitlab # [ 405.040304] postgres[4246]: [4246] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1825gitlab # [ 409.789601] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:50.558Z","message":"Booted Rails 7.2.3.1 application in production environment"}1826gitlab # [ 409.790030] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:50.558Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1827gitlab # [ 409.790222] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:50.558Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1828gitlab # [ 409.790345] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:50.558Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1829gitlab # [ 409.790461] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:50.560Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1830gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 73.24 seconds)1831gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1832gitlab # [ 410.545009] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:42:51 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1321833gitlab # [ 410.681582] gitaly[906]: time="2026-09-08T18:42:51.450Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M215909KTHN037GSEGYEQ7WD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:42:51.431" grpc.time_ms=18.444 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1834gitlab # Enumerating objects: 5, done.1835gitlab # [ 410.720902] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:42:51 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 1351836gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1837gitlab # Delta compression using up to 4 threads1838gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1839gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 136.00 KiB/s, done.1840gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1841gitlab # [ 411.310999] gitaly[906]: time="2026-09-08T18:42:52.080Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-08T18:43:22.060" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-08T18:42:52.060" grpc.time_ms=19.628 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 411.390992] gitaly[906]: time="2026-09-08T18:42:52.160Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M21590FN3NKH7N4TGS8VHMYS duration_ms=293ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/allowed"1843gitlab # [ 411.454356] gitaly[906]: time="2026-09-08T18:42:52.224Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M21590FN3NKH7N4TGS8VHMYS duration_ms=62ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/pre_receive"1844gitlab # [ 411.464463] gitaly[906]: time="2026-09-08T18:42:52.224Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:51.864" grpc.time_ms=360.326 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1845gitlab # [ 411.521586] gitaly[906]: time="2026-09-08T18:42:52.290Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:52.288" grpc.time_ms=1.659 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1846gitlab # [ 411.616637] gitaly[906]: time="2026-09-08T18:42:52.386Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:52.384" grpc.time_ms=2.215 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1847gitlab # [ 411.714577] gitaly[906]: time="2026-09-08T18:42:52.483Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:52.482" grpc.time_ms=0.875 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1848gitlab # [ 411.851795] gitaly[906]: time="2026-09-08T18:42:52.621Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M21590FN3NKH7N4TGS8VHMYS duration_ms=58ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/post_receive"1849gitlab # [ 411.861020] gitaly[906]: time="2026-09-08T18:42:52.622Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:52.562" grpc.time_ms=59.801 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1850gitlab # To http://gitlab/root/test-flake.git1851gitlab # da5a12e..bb589a2 master -> master1852gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.58 seconds)1853gitlab: must succeed: git -C /tmp/test-flake rev-parse master1854gitlab # [ 411.892632] gitaly[906]: time="2026-09-08T18:42:52.635Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS diskcache=034f8999-5cfb-42cf-9916-2820829dc214 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:42:51.710" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1855gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1856gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1857gitlab # [ 411.920237] gitlab-workhorse[823]: time="2026-09-08T18:42:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521858gitlab # [ 411.924792] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:42:52 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 11261859gitlab # [ 411.929354] gitaly[906]: time="2026-09-08T18:42:52.636Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=441 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13526 command.oublock=32 command.real_time_ms=918 command.spawn_token_fork_ms=2 command.system_time_ms=260 command.user_time_ms=181 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:42:51.710" grpc.time_ms=925.643 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1860gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.13 seconds)1861gitlab # [ 412.041138] nixbot[3531]: INFO: ::ffff:127.0.0.1:33888 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1862gitlab # [ 412.521364] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:53.289Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"545e887ba641b2029e343f58","created_at":"2026-09-08T18:41:24.135Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:24.220Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-545e887ba641b2029e343f58: start","job_status":"start","queue_duration_s":89.068668,"scheduling_latency_s":89.068761}1863gitlab # [ 412.903205] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:53.672Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"545e887ba641b2029e343f58","created_at":"2026-09-08T18:41:24.135Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:24.220Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-545e887ba641b2029e343f58: done: 0.382908 sec","job_status":"done","queue_duration_s":89.068668,"scheduling_latency_s":89.068761,"redis_calls":23,"redis_duration_s":0.010215,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.005422,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00269,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002103,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.205106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20399698,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.382908,"completed_at":"2026-09-08T18:42:53.671Z","load_balancing_strategy":"primary","db_duration_s":0.103865,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1864gitlab # [ 413.002253] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:53.675Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"74c61e8ac91c63c6d0f6aa4f","created_at":"2026-09-08T18:41:24.292Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:24.313Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-74c61e8ac91c63c6d0f6aa4f: start","job_status":"start","queue_duration_s":89.361665,"scheduling_latency_s":89.361703}1865gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1866gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.06 seconds)1867gitlab # [ 413.099644] nixbot[3531]: INFO: ::ffff:127.0.0.1:33896 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1868gitlab # [ 413.106209] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:53.872Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"74c61e8ac91c63c6d0f6aa4f","created_at":"2026-09-08T18:41:24.292Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:24.313Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-74c61e8ac91c63c6d0f6aa4f: done: 0.197324 sec","job_status":"done","queue_duration_s":89.361665,"scheduling_latency_s":89.361703,"redis_calls":14,"redis_duration_s":0.008673,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.003923,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00291,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00184,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.144064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1500412,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.197324,"completed_at":"2026-09-08T18:42:53.872Z","load_balancing_strategy":"primary","db_duration_s":0.038102,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1869gitlab # [ 413.187024] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:53.876Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"677ca3a58d8dbffaad9427f5","created_at":"2026-09-08T18:41:25.804Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:25.812Z","job_size_bytes":5,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-677ca3a58d8dbffaad9427f5: start","job_status":"start","queue_duration_s":88.063808,"scheduling_latency_s":88.063855}1870gitlab # [ 413.388799] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:54.158Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"677ca3a58d8dbffaad9427f5","created_at":"2026-09-08T18:41:25.804Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:25.812Z","job_size_bytes":5,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-677ca3a58d8dbffaad9427f5: done: 0.281486 sec","job_status":"done","queue_duration_s":88.063808,"scheduling_latency_s":88.063855,"redis_calls":12,"redis_duration_s":0.004288,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001362,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002289,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.228831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040616,"duration_s":0.281486,"completed_at":"2026-09-08T18:42:54.158Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042375,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1871gitlab # [ 413.487401] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:54.160Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"939a6ce8e2b41797e7ff3832","created_at":"2026-09-08T18:41:25.849Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:25.856Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-939a6ce8e2b41797e7ff3832: start","job_status":"start","queue_duration_s":88.304374,"scheduling_latency_s":88.304396}1872gitlab # [ 413.679554] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:54.448Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"939a6ce8e2b41797e7ff3832","created_at":"2026-09-08T18:41:25.849Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:25.856Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-939a6ce8e2b41797e7ff3832: done: 0.28761 sec","job_status":"done","queue_duration_s":88.304374,"scheduling_latency_s":88.304396,"redis_calls":11,"redis_duration_s":0.003283,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000794,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001888,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000601,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.242,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.242,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.185106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00057818,"duration_s":0.28761,"completed_at":"2026-09-08T18:42:54.447Z","load_balancing_strategy":"primary","db_duration_s":0.106154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1873gitlab # [ 413.774424] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:54.450Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"61cf07ec22e608f6f274ae42","created_at":"2026-09-08T18:41:25.860Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:25.865Z","job_size_bytes":89,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-61cf07ec22e608f6f274ae42: start","job_status":"start","queue_duration_s":88.584833,"scheduling_latency_s":88.584859}1874gitlab # [ 413.794873] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:54.498Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"61cf07ec22e608f6f274ae42","created_at":"2026-09-08T18:41:25.860Z","trace_propagation_headers":{"sentry-trace":"deff843991dd48e48d3f2b41477fb719-d81d2a0b8be24960","baggage":"sentry-trace_id=deff843991dd48e48d3f2b41477fb719,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M215684E54628ZQSJ28Q3QQW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:25.865Z","job_size_bytes":89,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-61cf07ec22e608f6f274ae42: done: 0.048284 sec","job_status":"done","queue_duration_s":88.584833,"scheduling_latency_s":88.584859,"redis_calls":11,"redis_duration_s":0.002957,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000789,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000546,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051778,"duration_s":0.048284,"completed_at":"2026-09-08T18:42:54.498Z","load_balancing_strategy":"primary","db_duration_s":0.008278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1875gitlab # [ 413.847841] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:54.500Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 da5a12ee3cffbf5b7dee97e408207083a88bbf1e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"856a1baa3b0454e7f3c38c5b","created_at":"2026-09-08T18:41:33.461Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-45e98002b4c94398","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:33.470Z","job_size_bytes":185,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-856a1baa3b0454e7f3c38c5b: start","job_status":"start","queue_duration_s":81.030445,"scheduling_latency_s":81.030467}1876gitlab # [ 413.867886] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:54.519Z","message":"changes: \"0000000000000000000000000000000000000000 da5a12ee3cffbf5b7dee97e408207083a88bbf1e refs/heads/master\\n\""}1877gitlab # [ 413.871020] gitaly[906]: time="2026-09-08T18:42:54.565Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:04.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:54.551" grpc.time_ms=13.301 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 413.884884] gitaly[906]: time="2026-09-08T18:42:54.588Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:43:04.020" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:42:54.570" grpc.time_ms=17.225 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 413.897654] gitaly[906]: time="2026-09-08T18:42:54.659Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=6 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-08T18:43:04.088" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:54.639" grpc.time_ms=19.772 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 413.939679] gitaly[906]: time="2026-09-08T18:42:54.709Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:54.709" grpc.time_ms=0.569 pid=906 span.kind=server system=grpc1881gitlab # [ 413.988786] gitaly[906]: time="2026-09-08T18:42:54.758Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:54.757" grpc.time_ms=0.792 pid=906 span.kind=server system=grpc1882gitlab # [ 414.004678] gitaly[906]: time="2026-09-08T18:42:54.773Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 diskcache=1de03754-7bf4-4c72-a66b-7cabba81e88e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T18:43:04.120" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:42:54.670" pid=906 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1883gitlab # [ 414.021112] gitaly[906]: time="2026-09-08T18:42:54.774Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=105 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4938 command.oublock=8 command.real_time_ms=96 command.spawn_token_fork_ms=0 command.system_time_ms=53 command.user_time_ms=52 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T18:43:04.120" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:42:54.670" grpc.time_ms=103.553 pid=906 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1884gitlab # [ 414.039026] gitaly[906]: time="2026-09-08T18:42:54.806Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T18:43:04.255" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:42:54.805" grpc.time_ms=0.342 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 414.069974] gitaly[906]: time="2026-09-08T18:42:54.840Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-08T18:43:04.264" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:54.815" grpc.time_ms=24.559 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1887gitlab # [ 414.169824] nixbot[3531]: INFO: ::ffff:127.0.0.1:33902 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1888gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.07 seconds)1889gitlab # [ 414.247486] gitaly[906]: time="2026-09-08T18:42:55.017Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-08T18:43:25.004" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:55.005" grpc.time_ms=12.171 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 414.304998] gitaly[906]: time="2026-09-08T18:42:55.074Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T18:43:05.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:42:55.074" grpc.time_ms=0.555 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891gitlab # [ 414.335493] gitaly[906]: time="2026-09-08T18:42:55.090Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-08T18:43:05.009" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-08T18:42:55.079" grpc.time_ms=11.052 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 414.366061] gitaly[906]: time="2026-09-08T18:42:55.136Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:05.048" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:55.118" grpc.time_ms=17.145 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893gitlab # [ 414.394775] gitaly[906]: time="2026-09-08T18:42:55.164Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:05.076" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:55.146" grpc.time_ms=17.991 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894gitlab # [ 414.448327] gitaly[906]: time="2026-09-08T18:42:55.218Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=17 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-08T18:43:25.189" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:55.189" grpc.time_ms=28.681 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1896gitlab # [ 415.165097] gitaly[906]: time="2026-09-08T18:42:55.933Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-08T18:43:05.000" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:55.931" grpc.time_ms=2.364 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.08 seconds)1898gitlab # [ 415.241991] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:56.009Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 da5a12ee3cffbf5b7dee97e408207083a88bbf1e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"856a1baa3b0454e7f3c38c5b","created_at":"2026-09-08T18:41:33.461Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-45e98002b4c94398","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:41:33.470Z","job_size_bytes":185,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-856a1baa3b0454e7f3c38c5b: done: 1.508942 sec","job_status":"done","queue_duration_s":81.030445,"scheduling_latency_s":81.030467,"gitaly_calls":13,"gitaly_duration_s":0.323509,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.06553199999999999,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.00691,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.009921,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002251,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005501,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.025702,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.015247,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.342,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.389,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.035822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00072968,"exclusive_lock_hold_duration_s":0.0346044,"duration_s":1.508942,"completed_at":"2026-09-08T18:42:56.009Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.153406,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1899gitlab # [ 415.339152] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:56.013Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7768e080091d3fd48e7316c2","created_at":"2026-09-08T18:41:40.216Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-87b6a5fb23fe4988","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:40.255Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7768e080091d3fd48e7316c2: start","job_status":"start","queue_duration_s":75.757342,"scheduling_latency_s":75.757385}1900gitlab # [ 415.357716] nixbot[3531]: INFO: ::ffff:127.0.0.1:33910 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1901gitlab # [ 415.401684] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:56.170Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7768e080091d3fd48e7316c2","created_at":"2026-09-08T18:41:40.216Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-87b6a5fb23fe4988","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:40.255Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7768e080091d3fd48e7316c2: done: 0.158086 sec","job_status":"done","queue_duration_s":75.757342,"scheduling_latency_s":75.757385,"redis_calls":16,"redis_duration_s":0.007393,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003314,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002448,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00075352,"duration_s":0.158086,"completed_at":"2026-09-08T18:42:56.170Z","load_balancing_strategy":"primary","db_duration_s":0.040284,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1902gitlab # [ 415.497750] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:56.173Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"688e1396d0ee2a2ef624e1a1","created_at":"2026-09-08T18:41:40.257Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-87b6a5fb23fe4988","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:40.264Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-688e1396d0ee2a2ef624e1a1: start","job_status":"start","queue_duration_s":75.909693,"scheduling_latency_s":75.909724}1903gitlab # [ 415.601125] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:56.369Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"688e1396d0ee2a2ef624e1a1","created_at":"2026-09-08T18:41:40.257Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-87b6a5fb23fe4988","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:40.264Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-688e1396d0ee2a2ef624e1a1: done: 0.196013 sec","job_status":"done","queue_duration_s":75.909693,"scheduling_latency_s":75.909724,"redis_calls":21,"redis_duration_s":0.008742,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003857,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002557,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002328,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.123693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00096348,"duration_s":0.196013,"completed_at":"2026-09-08T18:42:56.369Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1904gitlab # [ 415.701029] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:56.374Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"38629b5b9cc244255b3a27ae","created_at":"2026-09-08T18:41:40.434Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-87b6a5fb23fe4988","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:40.447Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-38629b5b9cc244255b3a27ae: start","job_status":"start","queue_duration_s":75.926286,"scheduling_latency_s":75.926335}1905gitlab # [ 416.100133] gitaly[906]: time="2026-09-08T18:42:56.870Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:56.869" grpc.time_ms=0.864 pid=906 span.kind=server system=grpc1906gitlab # [ 416.157513] gitaly[906]: time="2026-09-08T18:42:56.927Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:56.926" grpc.time_ms=0.854 pid=906 span.kind=server system=grpc1907gitlab # [ 416.213137] gitaly[906]: time="2026-09-08T18:42:56.982Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:42:56.981" grpc.time_ms=0.823 pid=906 span.kind=server system=grpc1908gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1909gitlab # [ 416.242500] gitaly[906]: time="2026-09-08T18:42:56.997Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN diskcache=707594db-8a94-41ed-948a-ce31b11ab5cc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T18:43:26.809" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:56.810" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab # [ 416.265841] gitaly[906]: time="2026-09-08T18:42:56.998Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=205 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7501 command.oublock=0 command.real_time_ms=180 command.spawn_token_fork_ms=1 command.system_time_ms=135 command.user_time_ms=70 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T18:43:26.809" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:56.810" grpc.time_ms=186.951 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab # [ 416.319238] nixbot[3531]: INFO: ::ffff:127.0.0.1:33924 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1912gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.07 seconds)1913gitlab # [ 416.437730] gitaly[906]: time="2026-09-08T18:42:57.207Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T18:43:27.205" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:57.205" grpc.time_ms=1.476 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 416.502150] gitaly[906]: time="2026-09-08T18:42:57.272Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156T834JP87B7DAM62CHWN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:27.269" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:57.270" grpc.time_ms=1.678 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab # [ 416.546661] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.314Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"38629b5b9cc244255b3a27ae","created_at":"2026-09-08T18:41:40.434Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-87b6a5fb23fe4988","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T18:41:40.447Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-38629b5b9cc244255b3a27ae: done: 0.940621 sec","job_status":"done","queue_duration_s":75.926286,"scheduling_latency_s":75.926335,"gitaly_calls":3,"gitaly_duration_s":0.201024,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.04982,"redis_read_bytes":455,"redis_write_bytes":27996,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002484,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002446,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.016271,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.00623,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14491,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.010511,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000746,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000931,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007717,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.101,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.05,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.056,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.053,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.597529,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.8104013600000001,"duration_s":0.940621,"completed_at":"2026-09-08T18:42:57.314Z","load_balancing_strategy":"primary","db_duration_s":0.119792,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1916gitlab # [ 416.640578] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.317Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","da5a12ee3cffbf5b7dee97e408207083a88bbf1e bb589a2d290709cbf570f8d4a4f14687f88c80e3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"bc25dfd7a81ee5590b108a49","created_at":"2026-09-08T18:42:52.599Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-53e684003af345b0","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:42:52.603Z","job_size_bytes":185,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-bc25dfd7a81ee5590b108a49: start","job_status":"start","queue_duration_s":4.714257,"scheduling_latency_s":4.714298}1917gitlab # [ 416.658229] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.341Z","message":"changes: \"da5a12ee3cffbf5b7dee97e408207083a88bbf1e bb589a2d290709cbf570f8d4a4f14687f88c80e3 refs/heads/master\\n\""}1918gitlab # [ 416.660649] gitaly[906]: time="2026-09-08T18:42:57.391Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:07.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:57.374" grpc.time_ms=16.979 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 416.673168] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.424Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f431de391714b458e6f12745","created_at":"2026-09-08T18:42:57.421Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-ab4f2647a36944bf","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"e2d761635bbacc5ba308b5d7","job_size_bytes":46,"pid":3399,"job_status":"deduplicated","message":"ProjectCacheWorker JID-f431de391714b458e6f12745: deduplicated: until executing","deduplication.type":"until executing"}1920gitlab # [ 416.688823] gitaly[906]: time="2026-09-08T18:42:57.416Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T18:43:07.030" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:42:57.400" grpc.time_ms=15.837 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921gitlab # [ 416.702905] gitaly[906]: time="2026-09-08T18:42:57.469Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-08T18:43:07.081" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:57.452" grpc.time_ms=17.368 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922gitlab # [ 416.727664] gitaly[906]: time="2026-09-08T18:42:57.497Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-08T18:43:27.477" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:57.478" grpc.time_ms=19.367 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab # [ 416.767496] gitaly[906]: time="2026-09-08T18:42:57.531Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-08T18:43:07.150" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-08T18:42:57.520" grpc.time_ms=10.446 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 416.794890] gitaly[906]: time="2026-09-08T18:42:57.564Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:07.177" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:57.548" grpc.time_ms=16.476 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab # [ 416.961619] gitaly[906]: time="2026-09-08T18:42:57.731Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-08T18:43:07.178" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:57.729" grpc.time_ms=2.25 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926gitlab # [ 417.000449] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.766Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","da5a12ee3cffbf5b7dee97e408207083a88bbf1e bb589a2d290709cbf570f8d4a4f14687f88c80e3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"bc25dfd7a81ee5590b108a49","created_at":"2026-09-08T18:42:52.599Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-53e684003af345b0","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T18:42:52.603Z","job_size_bytes":185,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-bc25dfd7a81ee5590b108a49: done: 0.448833 sec","job_status":"done","queue_duration_s":4.714257,"scheduling_latency_s":4.714298,"gitaly_calls":7,"gitaly_duration_s":0.128787,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.037558,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.003997,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001626,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005671,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.011909,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.014355,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.042,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.042,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.2747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00119314,"exclusive_lock_hold_duration_s":0.0340104,"duration_s":0.448833,"completed_at":"2026-09-08T18:42:57.766Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042283,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1927gitlab # [ 417.086224] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.769Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e2d761635bbacc5ba308b5d7","created_at":"2026-09-08T18:42:54.591Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:54.609Z","job_size_bytes":46,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e2d761635bbacc5ba308b5d7: start","job_status":"start","queue_duration_s":3.159881,"scheduling_latency_s":3.159917}1928gitlab # [ 417.103184] gitaly[906]: time="2026-09-08T18:42:57.846Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-09T00:42:57.835" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:42:57.836" grpc.time_ms=9.388 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 417.115355] gitaly[906]: time="2026-09-08T18:42:57.874Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-08T18:43:27.856" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:57.856" grpc.time_ms=17.218 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 417.144301] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.913Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e2d761635bbacc5ba308b5d7","created_at":"2026-09-08T18:42:54.591Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:54.609Z","job_size_bytes":46,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e2d761635bbacc5ba308b5d7: done: 0.144835 sec","job_status":"done","queue_duration_s":3.159881,"scheduling_latency_s":3.159917,"gitaly_calls":2,"gitaly_duration_s":0.03257,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010008,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001894,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000752,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002837,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003511,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048226,"duration_s":0.144835,"completed_at":"2026-09-08T18:42:57.913Z","load_balancing_strategy":"primary","db_duration_s":0.018128,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1931gitlab # [ 417.192835] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:57.916Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5674908abe2311546a011d97","created_at":"2026-09-08T18:42:54.629Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:54.635Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5674908abe2311546a011d97: start","job_status":"start","queue_duration_s":3.280775,"scheduling_latency_s":3.280798}1932gitlab # [ 417.214142] gitaly[906]: time="2026-09-08T18:42:57.983Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 diskcache=3b75421b-4627-477c-a64e-5c5a49da06ee grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-09T00:42:57.945" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:57.946" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933gitlab # [ 417.223890] gitaly[906]: time="2026-09-08T18:42:57.984Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=45 command.spawn_token_fork_ms=3 command.system_time_ms=25 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-09T00:42:57.945" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:57.946" grpc.time_ms=37.525 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1935gitlab # [ 417.277458] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:58.046Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5674908abe2311546a011d97","created_at":"2026-09-08T18:42:54.629Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:54.635Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5674908abe2311546a011d97: done: 0.130906 sec","job_status":"done","queue_duration_s":3.280775,"scheduling_latency_s":3.280798,"gitaly_calls":1,"gitaly_duration_s":0.040847,"redis_calls":15,"redis_duration_s":0.004515,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001833,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000759,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001449,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.032,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09663568,"duration_s":0.130906,"completed_at":"2026-09-08T18:42:58.046Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1936gitlab # [ 417.342523] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:58.049Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"040c3a3f10efe78f1fd1f25f","created_at":"2026-09-08T18:42:54.797Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:54.800Z","job_size_bytes":23,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-040c3a3f10efe78f1fd1f25f: start","job_status":"start","queue_duration_s":3.248338,"scheduling_latency_s":3.248363}1937gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.07 seconds)1938gitlab # [ 417.384995] gitaly[906]: time="2026-09-08T18:42:58.105Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-08T18:43:28.087" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:58.087" grpc.time_ms=17.195 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1939gitlab # [ 417.411429] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:58.122Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"040c3a3f10efe78f1fd1f25f","created_at":"2026-09-08T18:42:54.797Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:54.800Z","job_size_bytes":23,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-040c3a3f10efe78f1fd1f25f: done: 0.073539 sec","job_status":"done","queue_duration_s":3.248338,"scheduling_latency_s":3.248363,"gitaly_calls":1,"gitaly_duration_s":0.020234,"redis_calls":16,"redis_duration_s":0.009963,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000428,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002421,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.005559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073539,"completed_at":"2026-09-08T18:42:58.122Z","load_balancing_strategy":"primary","db_duration_s":0.004626,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1940gitlab # [ 417.478399] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:58.124Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e7cfe4d039bc98cfeeac740e","created_at":"2026-09-08T18:42:55.643Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b1c96a0c56d70d1ca401d2ec1cf4a84e801c53d7dd1c333b36f9906ec6c78195","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:55.658Z","job_size_bytes":467,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e7cfe4d039bc98cfeeac740e: start","job_status":"start","queue_duration_s":2.466545,"scheduling_latency_s":2.466568}1941gitlab # [ 417.496978] nixbot[3531]: INFO: ::ffff:127.0.0.1:33926 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1942gitlab # [ 417.498894] gitaly[906]: time="2026-09-08T18:42:58.173Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:28.171" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:58.172" grpc.time_ms=0.865 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1943gitlab # [ 417.510940] gitaly[906]: time="2026-09-08T18:42:58.190Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T18:43:08.107" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:58.178" grpc.time_ms=11.715 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1944gitlab # [ 417.604517] gitaly[906]: time="2026-09-08T18:42:58.374Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T18:43:28.372" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:58.372" grpc.time_ms=1.412 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945gitlab # [ 417.640243] gitaly[906]: time="2026-09-08T18:42:58.388Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=297 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-08T18:43:28.381" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:58.381" grpc.time_ms=6.061 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946gitlab # [ 417.794607] gitaly[906]: time="2026-09-08T18:42:58.564Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:08.178" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:42:58.548" grpc.time_ms=15.651 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1948gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.06 seconds)1949gitlab # [ 418.452966] nixbot[3531]: INFO: ::ffff:127.0.0.1:33928 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1950gitlab # [ 418.613173] gitaly[906]: time="2026-09-08T18:42:59.382Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=20 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-08T18:43:29.354" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:59.355" grpc.time_ms=27.218 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951gitlab # [ 418.643145] gitaly[906]: time="2026-09-08T18:42:59.413Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2156K21TJP6Z54ACQSD8DJ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-08T18:43:29.387" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:42:59.388" grpc.time_ms=24.495 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1952gitlab # [ 418.709807] sidekiq-cluster[3399]: {"severity":"WARN","time":"2026-09-08T18:42:59.478Z","class":"CreatePipelineWorker","correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"da5a12ee3cffbf5b7dee97e408207083a88bbf1e","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"da5a12ee3cffbf5b7dee97e408207083a88bbf1e","ref":"refs/heads/master"},"retry":0}1953gitlab # [ 418.727891] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.496Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e7cfe4d039bc98cfeeac740e","created_at":"2026-09-08T18:42:55.643Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b1c96a0c56d70d1ca401d2ec1cf4a84e801c53d7dd1c333b36f9906ec6c78195","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:55.658Z","job_size_bytes":467,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e7cfe4d039bc98cfeeac740e: done: 1.37171 sec","job_status":"done","queue_duration_s":2.466545,"scheduling_latency_s":2.466568,"gitaly_calls":7,"gitaly_duration_s":0.114671,"redis_calls":51,"redis_duration_s":0.016391,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.001501,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.006715,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002314,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004257,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.17496,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00065382,"duration_s":1.37171,"completed_at":"2026-09-08T18:42:59.496Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084247,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1954gitlab # [ 418.809217] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.500Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"c778b30f-de69-4a31-a919-82449d0365c0\"}"],"class":"WebHookWorker","jid":"2351469a9363a523610416e9","created_at":"2026-09-08T18:42:55.887Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a905e84e6269348fccf1e259545cecbc166dea9340a3f17e830b2994623b706","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:55.894Z","job_size_bytes":1751,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2351469a9363a523610416e9: start","job_status":"start","queue_duration_s":3.605719,"scheduling_latency_s":3.605757}1955gitlab # [ 418.940425] nixbot[3531]: INFO: ::ffff:127.0.0.1:33934 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1956gitlab # [ 418.981911] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.750Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"c778b30f-de69-4a31-a919-82449d0365c0\"}"],"class":"WebHookWorker","jid":"2351469a9363a523610416e9","created_at":"2026-09-08T18:42:55.887Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-3f0699d884154d6c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a905e84e6269348fccf1e259545cecbc166dea9340a3f17e830b2994623b706","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:55.894Z","job_size_bytes":1751,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2351469a9363a523610416e9: done: 0.249771 sec","job_status":"done","queue_duration_s":3.605719,"scheduling_latency_s":3.605757,"redis_calls":19,"redis_duration_s":0.007404,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001918,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000712,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003008,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001766,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0911891200000241,"cpu_s":0.134614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00074594,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.249771,"completed_at":"2026-09-08T18:42:59.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1957gitlab # [ 419.065598] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.754Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"487a545b72db0ef6b130c7c7","created_at":"2026-09-08T18:42:56.787Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:56.794Z","job_size_bytes":23,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-487a545b72db0ef6b130c7c7: start","job_status":"start","queue_duration_s":2.959876,"scheduling_latency_s":2.959971}1958gitlab # [ 419.101281] gitaly[906]: time="2026-09-08T18:42:59.871Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21598HNJ6FJDGTZHAM40DA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:42:59.858" grpc.time_ms=12.533 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1959gitlab # [ 419.129126] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:42:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1061960gitlab # [ 419.141300] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.910Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"487a545b72db0ef6b130c7c7","created_at":"2026-09-08T18:42:56.787Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:56.794Z","job_size_bytes":23,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-487a545b72db0ef6b130c7c7: done: 0.156642 sec","job_status":"done","queue_duration_s":2.959876,"scheduling_latency_s":2.959971,"redis_calls":10,"redis_duration_s":0.0029,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000647,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.117596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056854,"duration_s":0.156642,"completed_at":"2026-09-08T18:42:59.910Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026649,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1961gitlab # [ 419.215528] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.913Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b0444a9fcd40fcde4845ed08","created_at":"2026-09-08T18:42:57.007Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.020Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b0444a9fcd40fcde4845ed08: start","job_status":"start","queue_duration_s":2.892468,"scheduling_latency_s":2.892493}1962gitlab # [ 419.234195] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.962Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b0444a9fcd40fcde4845ed08","created_at":"2026-09-08T18:42:57.007Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.020Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b0444a9fcd40fcde4845ed08: done: 0.049537 sec","job_status":"done","queue_duration_s":2.892468,"scheduling_latency_s":2.892493,"redis_calls":10,"redis_duration_s":0.003624,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004624,"duration_s":0.049537,"completed_at":"2026-09-08T18:42:59.962Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1963gitlab # [ 419.278319] gitaly[906]: time="2026-09-08T18:43:00.011Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M21598N23RP2KBA19VDTSYS6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:42:59.995" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1964gitlab # [ 419.288494] gitlab-workhorse[823]: time="2026-09-08T18:43:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671965gitlab # [ 419.290302] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1381966gitlab # [ 419.292198] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:42:59.964Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b02647b87c256318d6a294a5","created_at":"2026-09-08T18:42:57.050Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.063Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b02647b87c256318d6a294a5: start","job_status":"start","queue_duration_s":2.90156,"scheduling_latency_s":2.90158}1967gitlab # [ 419.307876] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.024Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b02647b87c256318d6a294a5","created_at":"2026-09-08T18:42:57.050Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.063Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b02647b87c256318d6a294a5: done: 0.060028 sec","job_status":"done","queue_duration_s":2.90156,"scheduling_latency_s":2.90158,"redis_calls":10,"redis_duration_s":0.004352,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002395,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00143,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079344,"duration_s":0.060028,"completed_at":"2026-09-08T18:43:00.024Z","load_balancing_strategy":"primary","db_duration_s":0.011282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968gitlab # [ 419.351481] gitaly[906]: time="2026-09-08T18:43:00.011Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21598N23RP2KBA19VDTSYS6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:42:59.995" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1969gitlab # [ 419.364836] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.027Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8438447d695d1472595c8077","created_at":"2026-09-08T18:42:57.093Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.102Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8438447d695d1472595c8077: start","job_status":"start","queue_duration_s":2.924639,"scheduling_latency_s":2.924669}1970gitlab # [ 419.381970] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.093Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8438447d695d1472595c8077","created_at":"2026-09-08T18:42:57.093Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.102Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8438447d695d1472595c8077: done: 0.065802 sec","job_status":"done","queue_duration_s":2.924639,"scheduling_latency_s":2.924669,"redis_calls":13,"redis_duration_s":0.003699,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000516,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000782,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043476,"duration_s":0.065802,"completed_at":"2026-09-08T18:43:00.093Z","load_balancing_strategy":"primary","db_duration_s":0.012438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1971gitlab # [ 419.426548] gitlab-workhorse[823]: time="2026-09-08T18:43:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3361972gitlab # [ 419.428376] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 1131973gitlab # [ 419.430229] gitaly[906]: time="2026-09-08T18:43:00.011Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21598N23RP2KBA19VDTSYS6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:42:59.995" grpc.time_ms=15.7 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1974gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'1975gitlab # [ 419.444700] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.095Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"05aacdd49a5baed404c729a9","created_at":"2026-09-08T18:42:57.104Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.113Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-05aacdd49a5baed404c729a9: start","job_status":"start","queue_duration_s":2.982269,"scheduling_latency_s":2.982295}1976gitlab # [ 419.460900] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.146Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"05aacdd49a5baed404c729a9","created_at":"2026-09-08T18:42:57.104Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.113Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-05aacdd49a5baed404c729a9: done: 0.051292 sec","job_status":"done","queue_duration_s":2.982269,"scheduling_latency_s":2.982295,"redis_calls":10,"redis_duration_s":0.008033,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00691,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006689,"duration_s":0.051292,"completed_at":"2026-09-08T18:43:00.146Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1977gitlab # [ 419.503592] nixbot[3531]: {"ts": "2026-09-08T18:43:00.197Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1978gitlab # [ 419.506162] gitaly[906]: time="2026-09-08T18:43:00.128Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M21598SFX6F2SJ5C6SG09QRN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:00.097" pid=906 remote_ip="2001:db8:1::1" request_sha=174057feecb01fe88bd9b23de75d2b70d0556779 response_bytes=336 span.kind=server system=grpc1979gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.07 seconds)1980gitlab # [ 419.516111] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.148Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"b0412a9e627d12271a6f4a71","created_at":"2026-09-08T18:42:57.114Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.125Z","job_size_bytes":19,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-b0412a9e627d12271a6f4a71: start","job_status":"start","queue_duration_s":3.023211,"scheduling_latency_s":3.023229}1981gitlab # [ 419.532042] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"b0412a9e627d12271a6f4a71","created_at":"2026-09-08T18:42:57.114Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.125Z","job_size_bytes":19,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-b0412a9e627d12271a6f4a71: done: 0.043051 sec","job_status":"done","queue_duration_s":3.023211,"scheduling_latency_s":3.023229,"redis_calls":11,"redis_duration_s":0.004949,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001386,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026904,"duration_s":0.043051,"completed_at":"2026-09-08T18:43:00.191Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006756,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1982gitlab # [ 419.576576] gitaly[906]: time="2026-09-08T18:43:00.128Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=2 command.system_time_ms=18 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M21598SFX6F2SJ5C6SG09QRN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:00.097" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1983gitlab # [ 419.590312] nixbot[3531]: INFO: ::ffff:127.0.0.1:33950 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK1984gitlab # [ 419.592234] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.193Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"882f2451e0fb1b9ba4309d3c","created_at":"2026-09-08T18:42:57.129Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.138Z","job_size_bytes":66,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-882f2451e0fb1b9ba4309d3c: start","job_status":"start","queue_duration_s":3.054967,"scheduling_latency_s":3.054986}1985gitlab # [ 419.609334] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.238Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"882f2451e0fb1b9ba4309d3c","created_at":"2026-09-08T18:42:57.129Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.138Z","job_size_bytes":66,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-882f2451e0fb1b9ba4309d3c: done: 0.04551 sec","job_status":"done","queue_duration_s":3.054967,"scheduling_latency_s":3.054986,"redis_calls":10,"redis_duration_s":0.0038480000000000003,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00039,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002471,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00080612,"duration_s":0.04551,"completed_at":"2026-09-08T18:43:00.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1986gitlab # [ 419.654474] gitaly[906]: time="2026-09-08T18:43:00.129Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=2 command.system_time_ms=18 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M21598SFX6F2SJ5C6SG09QRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:00.097" grpc.time_ms=30.913 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc1987gitlab # [ 419.669006] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.240Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"bc4b65bd45a9c414d600667c","created_at":"2026-09-08T18:42:57.139Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.146Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bc4b65bd45a9c414d600667c: start","job_status":"start","queue_duration_s":3.094163,"scheduling_latency_s":3.094182}1988gitlab # [ 419.684973] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.310Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"bc4b65bd45a9c414d600667c","created_at":"2026-09-08T18:42:57.139Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.146Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bc4b65bd45a9c414d600667c: done: 0.069807 sec","job_status":"done","queue_duration_s":3.094163,"scheduling_latency_s":3.094182,"redis_calls":10,"redis_duration_s":0.0070220000000000005,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001759,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003388,"duration_s":0.069807,"completed_at":"2026-09-08T18:43:00.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01277,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1989gitlab # [ 419.728846] gitaly[906]: time="2026-09-08T18:43:00.334Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:30.332" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.332" grpc.time_ms=1.294 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 419.741825] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.315Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"aa10058b0174925ee9c4715f","created_at":"2026-09-08T18:42:57.156Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.163Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-aa10058b0174925ee9c4715f: start","job_status":"start","queue_duration_s":3.151956,"scheduling_latency_s":3.151974}1991gitlab # [ 419.758124] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.358Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"aa10058b0174925ee9c4715f","created_at":"2026-09-08T18:42:57.156Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.163Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-aa10058b0174925ee9c4715f: done: 0.042672 sec","job_status":"done","queue_duration_s":3.151956,"scheduling_latency_s":3.151974,"redis_calls":10,"redis_duration_s":0.004074,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000395,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002399,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032874,"duration_s":0.042672,"completed_at":"2026-09-08T18:43:00.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1992gitlab # [ 419.803171] gitaly[906]: time="2026-09-08T18:43:00.489Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:10.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:00.469" grpc.time_ms=19.078 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1993gitlab # [ 419.816609] nixbot[3531]: {"ts": "2026-09-08T18:43:00.572Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e \"HTTP/1.1 201 Created\""}1994gitlab # [ 419.819320] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.359Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91df8984531eee05cf573266","created_at":"2026-09-08T18:42:57.164Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.169Z","job_size_bytes":6,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91df8984531eee05cf573266: start","job_status":"start","queue_duration_s":3.190144,"scheduling_latency_s":3.190162}1995gitlab # [ 419.835125] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.383Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91df8984531eee05cf573266","created_at":"2026-09-08T18:42:57.164Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.169Z","job_size_bytes":6,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91df8984531eee05cf573266: done: 0.023591 sec","job_status":"done","queue_duration_s":3.190144,"scheduling_latency_s":3.190162,"redis_calls":10,"redis_duration_s":0.002547,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001506,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034958,"duration_s":0.023591,"completed_at":"2026-09-08T18:43:00.383Z","load_balancing_strategy":"primary","db_duration_s":0.002025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1996gitlab # [ 419.878376] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3471997gitlab # [ 419.880919] gitaly[906]: time="2026-09-08T18:43:00.553Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS diskcache=dba8027c-a45b-4ae6-a2af-544fd0f5bec8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-09T00:43:00.542" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.542" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998gitlab # [ 419.890715] gitaly[906]: time="2026-09-08T18:43:00.553Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-09T00:43:00.542" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.542" grpc.time_ms=10.183 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999gitlab # [ 419.903653] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.385Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a64b9ce90f1ac0dcdb472ec3","created_at":"2026-09-08T18:42:57.170Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.175Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a64b9ce90f1ac0dcdb472ec3: start","job_status":"start","queue_duration_s":3.209281,"scheduling_latency_s":3.209299}2000gitlab # [ 419.919023] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.428Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a64b9ce90f1ac0dcdb472ec3","created_at":"2026-09-08T18:42:57.170Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.175Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a64b9ce90f1ac0dcdb472ec3: done: 0.043021 sec","job_status":"done","queue_duration_s":3.209281,"scheduling_latency_s":3.209299,"redis_calls":10,"redis_duration_s":0.003838,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000501,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001857,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046682,"duration_s":0.043021,"completed_at":"2026-09-08T18:43:00.428Z","load_balancing_strategy":"primary","db_duration_s":0.010761,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001gitlab # [ 419.961760] gitaly[906]: time="2026-09-08T18:43:00.606Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:30.604" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.605" grpc.time_ms=0.636 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2002gitlab # [ 419.974042] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.430Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a738ea04705dac81c4fbc3f6","created_at":"2026-09-08T18:42:57.184Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.191Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a738ea04705dac81c4fbc3f6: start","job_status":"start","queue_duration_s":3.239088,"scheduling_latency_s":3.239123}2003gitlab # [ 419.989983] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.514Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a738ea04705dac81c4fbc3f6","created_at":"2026-09-08T18:42:57.184Z","trace_propagation_headers":{"sentry-trace":"b24240170d104d09b61a699bcb71950a-0d9411976ff84cc8","baggage":"sentry-trace_id=b24240170d104d09b61a699bcb71950a,sentry-release=Unknown"},"correlation_id":"01M2156T834JP87B7DAM62CHWN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.191Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a738ea04705dac81c4fbc3f6: done: 0.084075 sec","job_status":"done","queue_duration_s":3.239088,"scheduling_latency_s":3.239123,"redis_calls":14,"redis_duration_s":0.004021,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00099688,"duration_s":0.084075,"completed_at":"2026-09-08T18:43:00.514Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 420.034788] gitaly[906]: time="2026-09-08T18:43:00.620Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T18:43:10.238" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:00.608" grpc.time_ms=11.356 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2005gitlab # [ 420.047849] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.515Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c9e8bb53f3670892064a7d20","created_at":"2026-09-08T18:42:57.438Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-ab4f2647a36944bf","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"5674908abe2311546a011d97","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.443Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c9e8bb53f3670892064a7d20: start","job_status":"start","queue_duration_s":3.072339,"scheduling_latency_s":3.072361}2006gitlab # [ 420.064900] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.570Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c9e8bb53f3670892064a7d20","created_at":"2026-09-08T18:42:57.438Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-ab4f2647a36944bf","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"5674908abe2311546a011d97","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.443Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c9e8bb53f3670892064a7d20: done: 0.054379 sec","job_status":"done","queue_duration_s":3.072339,"scheduling_latency_s":3.072361,"gitaly_calls":1,"gitaly_duration_s":0.013946,"redis_calls":12,"redis_duration_s":0.004591,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00274,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000921,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0312781,"duration_s":0.054379,"completed_at":"2026-09-08T18:43:00.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007gitlab # [ 420.109528] gitaly[906]: time="2026-09-08T18:43:00.659Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T18:43:30.658" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.658" grpc.time_ms=1.024 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2008gitlab # [ 420.122420] nixbot[3531]: {"ts": "2026-09-08T18:43:00.856Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e \"HTTP/1.1 201 Created\""}2009gitlab # [ 420.125202] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:00.576Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"4365f407f8678c92fa94dc59","created_at":"2026-09-08T18:42:57.668Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-ab4f2647a36944bf","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1e9ecd41875d1a8e9e2c1c40c19d67d7787ff94b1ab1e2eadf1be6ac4b7534d3","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.673Z","job_size_bytes":467,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-4365f407f8678c92fa94dc59: start","job_status":"start","queue_duration_s":2.903033,"scheduling_latency_s":2.903074}2010gitlab # [ 420.142314] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2782011gitlab # [ 420.144801] gitaly[906]: time="2026-09-08T18:43:00.668Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=151 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-08T18:43:30.663" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.663" grpc.time_ms=3.892 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012gitlab # [ 420.160746] gitaly[906]: time="2026-09-08T18:43:00.677Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21599B0JA8ZH4H932FBPQ4J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:30.675" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.676" grpc.time_ms=1.042 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2013gitlab # [ 420.173277] gitaly[906]: time="2026-09-08T18:43:00.712Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=6 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:10.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:00.697" grpc.time_ms=15.126 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2014gitlab # [ 420.186122] gitaly[906]: time="2026-09-08T18:43:00.765Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M21599B0JA8ZH4H932FBPQ4J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:10.286" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:00.747" grpc.time_ms=17.955 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2015gitlab # [ 420.199173] gitaly[906]: time="2026-09-08T18:43:00.932Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21599KZCMQQDH1XHR9HHEPA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:30.831" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:00.931" grpc.time_ms=0.758 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2016gitlab # [ 420.230141] gitaly[906]: time="2026-09-08T18:43:01.000Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M21599KZCMQQDH1XHR9HHEPA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:10.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:00.982" grpc.time_ms=17.989 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2017gitlab # [ 420.295789] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2012018gitlab # [ 420.301652] nixbot[3531]: {"ts": "2026-09-08T18:43:01.066Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/da5a12ee3cffbf5b7dee97e408207083a88bbf1e \"HTTP/1.1 201 Created\""}2019gitlab # [ 420.476477] gitaly[906]: time="2026-09-08T18:43:01.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21599VYZ6JA23HGY2M1TBFP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.227" grpc.time_ms=18.805 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2020gitlab # [ 420.514922] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1282021gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'2022gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.09 seconds)2023gitlab # [ 420.607532] gitaly[906]: time="2026-09-08T18:43:01.377Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-08T18:43:31.057" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:01.358" grpc.time_ms=19.077 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024gitlab # [ 420.648984] gitaly[906]: time="2026-09-08T18:43:01.381Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2159A020F1X22P4AVEH94X6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.366" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2025gitlab # [ 420.667208] sidekiq-cluster[3399]: {"severity":"WARN","time":"2026-09-08T18:43:01.406Z","class":"CreatePipelineWorker","correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"da5a12ee3cffbf5b7dee97e408207083a88bbf1e","after":"bb589a2d290709cbf570f8d4a4f14687f88c80e3","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"bb589a2d290709cbf570f8d4a4f14687f88c80e3","ref":"refs/heads/master"},"retry":0}2026gitlab # [ 420.678256] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.412Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"4365f407f8678c92fa94dc59","created_at":"2026-09-08T18:42:57.668Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-ab4f2647a36944bf","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1e9ecd41875d1a8e9e2c1c40c19d67d7787ff94b1ab1e2eadf1be6ac4b7534d3","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.673Z","job_size_bytes":467,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-4365f407f8678c92fa94dc59: done: 0.835959 sec","job_status":"done","queue_duration_s":2.903033,"scheduling_latency_s":2.903074,"gitaly_calls":7,"gitaly_duration_s":0.080016,"redis_calls":33,"redis_duration_s":0.004939,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000613,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002425,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000806,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.733317,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.835959,"completed_at":"2026-09-08T18:43:01.412Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023557,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2027gitlab # [ 420.743205] nixbot[3531]: INFO: ::ffff:127.0.0.1:33954 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK2028gitlab # [ 420.745842] nixbot[3531]: INFO: ::ffff:127.0.0.1:33958 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2029gitlab # [ 420.747396] gitlab-workhorse[823]: time="2026-09-08T18:43:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672030gitlab # [ 420.749716] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1432031gitlab # [ 420.752379] gitaly[906]: time="2026-09-08T18:43:01.381Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159A020F1X22P4AVEH94X6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.366" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2032gitlab # [ 420.767111] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.413Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"6a69c665-51d1-4d1f-97f3-e76acbf2f0f1\"}"],"class":"WebHookWorker","jid":"4c30bd6dc8bc24ff7693e543","created_at":"2026-09-08T18:42:57.701Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-ab4f2647a36944bf","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3b193ad8abee01f737fb17f52a4e72cebaf7d9e240b14f32c4239c332e942505","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.705Z","job_size_bytes":1749,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4c30bd6dc8bc24ff7693e543: start","job_status":"start","queue_duration_s":3.708383,"scheduling_latency_s":3.708404}2033gitlab # [ 420.785015] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.459Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"6a69c665-51d1-4d1f-97f3-e76acbf2f0f1\"}"],"class":"WebHookWorker","jid":"4c30bd6dc8bc24ff7693e543","created_at":"2026-09-08T18:42:57.701Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-ab4f2647a36944bf","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3b193ad8abee01f737fb17f52a4e72cebaf7d9e240b14f32c4239c332e942505","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.705Z","job_size_bytes":1749,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4c30bd6dc8bc24ff7693e543: done: 0.045501 sec","job_status":"done","queue_duration_s":3.708383,"scheduling_latency_s":3.708404,"redis_calls":14,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000224,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.012945100000024468,"cpu_s":0.025288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.045501,"completed_at":"2026-09-08T18:43:01.459Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034gitlab # [ 420.833256] gitaly[906]: time="2026-09-08T18:43:01.381Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159A020F1X22P4AVEH94X6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.366" grpc.time_ms=14.631 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2035gitlab # [ 420.847678] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 652036gitlab # [ 420.849872] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.460Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"9b930a44226d6ccfccd08a0f","created_at":"2026-09-08T18:42:57.888Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-fcf07b59e5ed4489","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.895Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-9b930a44226d6ccfccd08a0f: start","job_status":"start","queue_duration_s":3.564508,"scheduling_latency_s":3.564524}2037gitlab # [ 420.865422] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.510Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"9b930a44226d6ccfccd08a0f","created_at":"2026-09-08T18:42:57.888Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-fcf07b59e5ed4489","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:57.895Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-9b930a44226d6ccfccd08a0f: done: 0.05022 sec","job_status":"done","queue_duration_s":3.564508,"scheduling_latency_s":3.564524,"redis_calls":19,"redis_duration_s":0.00296,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000613,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000405,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001472,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008183600000000001,"duration_s":0.05022,"completed_at":"2026-09-08T18:43:01.510Z","load_balancing_strategy":"primary","db_duration_s":0.005881,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038gitlab # [ 420.910071] gitaly[906]: time="2026-09-08T18:43:01.391Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M21590FN3NKH7N4TGS8VHMYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-08T18:43:31.080" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:01.381" grpc.time_ms=10.234 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2039gitlab # [ 420.923171] gitlab-workhorse[823]: time="2026-09-08T18:43:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672040gitlab # [ 420.925068] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 802041gitlab # [ 420.926951] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.512Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2351469a9363a523610416e9"],"class":"WebHooks::LogExecutionWorker","jid":"b89fb0cdb29b388166567a13","created_at":"2026-09-08T18:42:59.714Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-80bc48e8549d4158","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bbc95345fa48e69e6ea8764355df704db7b529f5c4531704b1ccf7f956d386f2","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:59.729Z","job_size_bytes":2593,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b89fb0cdb29b388166567a13: start","job_status":"start","queue_duration_s":1.782896,"scheduling_latency_s":1.782913}2042gitlab # [ 420.943747] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.557Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2351469a9363a523610416e9"],"class":"WebHooks::LogExecutionWorker","jid":"b89fb0cdb29b388166567a13","created_at":"2026-09-08T18:42:59.714Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-80bc48e8549d4158","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bbc95345fa48e69e6ea8764355df704db7b529f5c4531704b1ccf7f956d386f2","size_limiter":"validated","enqueued_at":"2026-09-08T18:42:59.729Z","job_size_bytes":2593,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b89fb0cdb29b388166567a13: done: 0.045135 sec","job_status":"done","queue_duration_s":1.782896,"scheduling_latency_s":1.782913,"redis_calls":10,"redis_duration_s":0.002667,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048954,"duration_s":0.045135,"completed_at":"2026-09-08T18:43:01.557Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043gitlab # [ 420.988663] gitaly[906]: time="2026-09-08T18:43:01.567Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2159A7X7C6A1A713TRBQHTG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.557" grpc.time_ms=9.619 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2044gitlab # [ 421.001279] gitaly[906]: time="2026-09-08T18:43:01.649Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2159AA142EZY5ND0EA5CPZY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.640" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2045gitlab # [ 421.011435] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.558Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5b0088c17f2e7080b635ab43","created_at":"2026-09-08T18:43:00.528Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-2b86bebd477d4e10","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:00.538Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5b0088c17f2e7080b635ab43: start","job_status":"start","queue_duration_s":1.020044,"scheduling_latency_s":1.02006}2046gitlab # [ 421.028280] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 632047gitlab # [ 421.030312] gitaly[906]: time="2026-09-08T18:43:01.649Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AA142EZY5ND0EA5CPZY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.640" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2048gitlab # [ 421.043292] gitaly[906]: time="2026-09-08T18:43:01.649Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AA142EZY5ND0EA5CPZY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.640" grpc.time_ms=8.996 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2049gitlab # [ 421.057706] gitaly[906]: time="2026-09-08T18:43:01.663Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:43:01.663" grpc.time_ms=0.439 pid=906 span.kind=server system=grpc2050gitlab # [ 421.067288] gitaly[906]: time="2026-09-08T18:43:01.691Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:43:01.691" grpc.time_ms=0.297 pid=906 span.kind=server system=grpc2051gitlab # [ 421.077760] gitaly[906]: time="2026-09-08T18:43:01.716Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:43:01.715" grpc.time_ms=0.414 pid=906 span.kind=server system=grpc2052gitlab # [ 421.087228] gitaly[906]: time="2026-09-08T18:43:01.726Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ diskcache=528c8c65-13ed-4644-838e-4983e6e063fd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T18:43:31.330" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:01.630" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053gitlab # [ 421.096965] gitlab-workhorse[823]: time="2026-09-08T18:43:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162054gitlab # [ 421.098689] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 842055gitlab # [ 421.100578] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.832Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5b0088c17f2e7080b635ab43","created_at":"2026-09-08T18:43:00.528Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-2b86bebd477d4e10","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:00.538Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3399,"job_status":"rescheduled","message":"PipelineProcessWorker JID-5b0088c17f2e7080b635ab43: rescheduled"}2056gitlab # [ 421.115221] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.841Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5b0088c17f2e7080b635ab43","created_at":"2026-09-08T18:43:00.528Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-2b86bebd477d4e10","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:00.538Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5b0088c17f2e7080b635ab43: done: 0.283358 sec","job_status":"done","queue_duration_s":1.020044,"scheduling_latency_s":1.02006,"gitaly_calls":3,"gitaly_duration_s":0.103428,"redis_calls":67,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.020525,"redis_read_bytes":149,"redis_write_bytes":18096,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000482,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00132,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003137,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.003602,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00315,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001937,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.006415,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.14041,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.24092254,"duration_s":0.283358,"completed_at":"2026-09-08T18:43:01.841Z","load_balancing_strategy":"primary","db_duration_s":0.028828,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2057gitlab # [ 421.165980] gitaly[906]: time="2026-09-08T18:43:01.726Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=85 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7139 command.oublock=0 command.real_time_ms=90 command.spawn_token_fork_ms=1 command.system_time_ms=26 command.user_time_ms=59 component=gitaly.UnaryServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T18:43:31.330" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:01.630" grpc.time_ms=95.545 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058gitlab # [ 421.178880] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.843Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4c30bd6dc8bc24ff7693e543"],"class":"WebHooks::LogExecutionWorker","jid":"acd5b1a7962bad4c6fe8cd55","created_at":"2026-09-08T18:43:01.451Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-cdf49661729d452d","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:05df93a1197ff1f22a5694004c0548781d3096c345dd84d5234df4dc91367858","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.453Z","job_size_bytes":2592,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-acd5b1a7962bad4c6fe8cd55: start","job_status":"start","queue_duration_s":0.390181,"scheduling_latency_s":0.390207}2059gitlab # [ 421.195610] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.876Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4c30bd6dc8bc24ff7693e543"],"class":"WebHooks::LogExecutionWorker","jid":"acd5b1a7962bad4c6fe8cd55","created_at":"2026-09-08T18:43:01.451Z","trace_propagation_headers":{"sentry-trace":"e5eb9bba6b85429e9810bb2e396c9fec-cdf49661729d452d","baggage":"sentry-trace_id=e5eb9bba6b85429e9810bb2e396c9fec,sentry-release=Unknown"},"correlation_id":"01M21590FN3NKH7N4TGS8VHMYS","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:05df93a1197ff1f22a5694004c0548781d3096c345dd84d5234df4dc91367858","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.453Z","job_size_bytes":2592,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-acd5b1a7962bad4c6fe8cd55: done: 0.032463 sec","job_status":"done","queue_duration_s":0.390181,"scheduling_latency_s":0.390207,"redis_calls":7,"redis_duration_s":0.004014,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001549,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032463,"completed_at":"2026-09-08T18:43:01.876Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2060gitlab # [ 421.239248] gitaly[906]: time="2026-09-08T18:43:01.749Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2159ADNMSR2ZNGFDNEK8XFF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=202 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.737" grpc.time_ms=11.28 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2061gitlab # [ 421.252248] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d6e69ef6c3a21ae43a7ca59a","created_at":"2026-09-08T18:43:01.496Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-55ca990eb085467c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.501Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d6e69ef6c3a21ae43a7ca59a: start","job_status":"start","queue_duration_s":0.376635,"scheduling_latency_s":0.376656}2062gitlab # [ 421.268671] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.976Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d6e69ef6c3a21ae43a7ca59a","created_at":"2026-09-08T18:43:01.496Z","trace_propagation_headers":{"sentry-trace":"bbecac5db1744a8e80f2bfee26b18afe-55ca990eb085467c","baggage":"sentry-trace_id=bbecac5db1744a8e80f2bfee26b18afe,sentry-release=Unknown"},"correlation_id":"01M2156K21TJP6Z54ACQSD8DJ8","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.501Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d6e69ef6c3a21ae43a7ca59a: done: 0.098055 sec","job_status":"done","queue_duration_s":0.376635,"scheduling_latency_s":0.376656,"redis_calls":11,"redis_duration_s":0.002646,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000516,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001806,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000324,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024676,"duration_s":0.098055,"completed_at":"2026-09-08T18:43:01.976Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 421.312981] gitaly[906]: time="2026-09-08T18:43:01.783Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T18:43:31.482" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:01.782" grpc.time_ms=0.708 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064gitlab # [ 421.325581] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:01.978Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d27b2529f28da35aad76c17d","created_at":"2026-09-08T18:43:01.622Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.624Z","job_size_bytes":23,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d27b2529f28da35aad76c17d: start","job_status":"start","queue_duration_s":0.353583,"scheduling_latency_s":0.353605}2065gitlab # [ 421.342347] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.021Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d27b2529f28da35aad76c17d","created_at":"2026-09-08T18:43:01.622Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.624Z","job_size_bytes":23,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d27b2529f28da35aad76c17d: done: 0.043584 sec","job_status":"done","queue_duration_s":0.353583,"scheduling_latency_s":0.353605,"redis_calls":7,"redis_duration_s":0.0012200000000000002,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043584,"completed_at":"2026-09-08T18:43:02.021Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008957,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2066gitlab # [ 421.385079] gitaly[906]: time="2026-09-08T18:43:01.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M21598ZYA1KZT9AB3EATG3QZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:31.520" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:01.820" grpc.time_ms=0.705 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 421.397396] gitaly[906]: time="2026-09-08T18:43:01.835Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2159AFQMENP8MSEDNGXYNAY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.818" pid=906 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2068gitlab # [ 421.407576] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.022Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5822c91590f1f2195ce2b5c2","created_at":"2026-09-08T18:43:01.731Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.733Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5822c91590f1f2195ce2b5c2: start","job_status":"start","queue_duration_s":0.289428,"scheduling_latency_s":0.289445}2069gitlab # [ 421.424028] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.061Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5822c91590f1f2195ce2b5c2","created_at":"2026-09-08T18:43:01.731Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.733Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5822c91590f1f2195ce2b5c2: done: 0.038607 sec","job_status":"done","queue_duration_s":0.289428,"scheduling_latency_s":0.289445,"redis_calls":7,"redis_duration_s":0.0017130000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038607,"completed_at":"2026-09-08T18:43:02.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004386,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 421.467967] gitaly[906]: time="2026-09-08T18:43:01.835Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AFQMENP8MSEDNGXYNAY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.818" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2071gitlab # [ 421.481802] nixbot[3531]: {"ts": "2026-09-08T18:43:02.205Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 \"HTTP/1.1 201 Created\""}2072gitlab # [ 421.484550] nixbot[3531]: {"ts": "2026-09-08T18:43:02.214Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-bb589a2d2907-5"}2073gitlab # [ 421.486853] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.063Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"55008785a8b962199d10a120","created_at":"2026-09-08T18:43:01.744Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.748Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-55008785a8b962199d10a120: start","job_status":"start","queue_duration_s":0.315633,"scheduling_latency_s":0.315663}2074gitlab # [ 421.502353] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.109Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"55008785a8b962199d10a120","created_at":"2026-09-08T18:43:01.744Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.748Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-55008785a8b962199d10a120: done: 0.045553 sec","job_status":"done","queue_duration_s":0.315633,"scheduling_latency_s":0.315663,"redis_calls":7,"redis_duration_s":0.002465,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045553,"completed_at":"2026-09-08T18:43:02.109Z","load_balancing_strategy":"primary","db_duration_s":0.006549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075gitlab # [ 421.544124] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3352076gitlab # [ 421.546572] gitaly[906]: time="2026-09-08T18:43:01.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AFQMENP8MSEDNGXYNAY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T18:43:01.818" grpc.time_ms=16.995 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc2077gitlab # [ 421.561209] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.111Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f96f3e89fc9434fa5b95de89","created_at":"2026-09-08T18:43:01.757Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.759Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f96f3e89fc9434fa5b95de89: start","job_status":"start","queue_duration_s":0.352321,"scheduling_latency_s":0.352359}2078gitlab # [ 421.576229] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.154Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f96f3e89fc9434fa5b95de89","created_at":"2026-09-08T18:43:01.757Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.759Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f96f3e89fc9434fa5b95de89: done: 0.043211 sec","job_status":"done","queue_duration_s":0.352321,"scheduling_latency_s":0.352359,"redis_calls":10,"redis_duration_s":0.002326,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001332,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000573,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043211,"completed_at":"2026-09-08T18:43:02.154Z","load_balancing_strategy":"primary","db_duration_s":0.006238,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2079gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3'2080gitlab # [ 421.619495] gitaly[906]: time="2026-09-08T18:43:01.935Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:31.834" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:01.934" grpc.time_ms=0.501 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2081gitlab # [ 421.631609] nixbot[3531]: {"ts": "2026-09-08T18:43:02.349Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-bb589a2d2907-5", "argv0": "bwrap"}2082gitlab # [ 421.633967] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.157Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f9f69b969e5d01143f5e6cb9","created_at":"2026-09-08T18:43:01.759Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.761Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f9f69b969e5d01143f5e6cb9: start","job_status":"start","queue_duration_s":0.395916,"scheduling_latency_s":0.395946}2083gitlab # [ 421.650346] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.192Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f9f69b969e5d01143f5e6cb9","created_at":"2026-09-08T18:43:01.759Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.761Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f9f69b969e5d01143f5e6cb9: done: 0.035027 sec","job_status":"done","queue_duration_s":0.395916,"scheduling_latency_s":0.395946,"redis_calls":7,"redis_duration_s":0.001373,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000386,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035027,"completed_at":"2026-09-08T18:43:02.192Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084gitlab # [ 421.692210] gitaly[906]: time="2026-09-08T18:43:01.966Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-08T18:43:31.850" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:01.950" grpc.time_ms=15.757 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2085gitlab # [ 421.705875] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0c22ba7d4789a207fd897ebf","created_at":"2026-09-08T18:43:01.761Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.763Z","job_size_bytes":19,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0c22ba7d4789a207fd897ebf: start","job_status":"start","queue_duration_s":0.429933,"scheduling_latency_s":0.429952}2086gitlab # [ 421.722005] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0c22ba7d4789a207fd897ebf","created_at":"2026-09-08T18:43:01.761Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.763Z","job_size_bytes":19,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0c22ba7d4789a207fd897ebf: done: 0.031769 sec","job_status":"done","queue_duration_s":0.429933,"scheduling_latency_s":0.429952,"redis_calls":8,"redis_duration_s":0.002073,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001318,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031769,"completed_at":"2026-09-08T18:43:02.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 421.764450] gitaly[906]: time="2026-09-08T18:43:01.985Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:11.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:01.973" grpc.time_ms=11.55 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088gitlab # [ 421.778061] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.226Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3b6cd181e36b7fc95b208c4e","created_at":"2026-09-08T18:43:01.764Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.766Z","job_size_bytes":66,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3b6cd181e36b7fc95b208c4e: start","job_status":"start","queue_duration_s":0.460505,"scheduling_latency_s":0.460524}2089gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3', in 0.19 seconds)2090(finished: subtest: push delivers the webhook, in 84.72 seconds)2091subtest: eval and build post commit statuses2092gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2093gitlab # [ 421.798490] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.261Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3b6cd181e36b7fc95b208c4e","created_at":"2026-09-08T18:43:01.764Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.766Z","job_size_bytes":66,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3b6cd181e36b7fc95b208c4e: done: 0.034883 sec","job_status":"done","queue_duration_s":0.460505,"scheduling_latency_s":0.460524,"redis_calls":7,"redis_duration_s":0.002157,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001518,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034883,"completed_at":"2026-09-08T18:43:02.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2094gitlab # [ 421.841298] gitaly[906]: time="2026-09-08T18:43:02.015Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T18:43:12.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:43:02.015" grpc.time_ms=0.281 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2095gitlab # [ 421.852907] nixbot[3531]: INFO: ::ffff:127.0.0.1:46934 - "GET /api/repos/gitlab/root/test-flake/builds?commit=bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 200 OK2096gitlab # [ 421.854822] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.263Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"34ff43faf16e14d6776bb769","created_at":"2026-09-08T18:43:01.766Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.768Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-34ff43faf16e14d6776bb769: start","job_status":"start","queue_duration_s":0.49528,"scheduling_latency_s":0.495299}2097gitlab # [ 421.871335] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.301Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"34ff43faf16e14d6776bb769","created_at":"2026-09-08T18:43:01.766Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.768Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-34ff43faf16e14d6776bb769: done: 0.037798 sec","job_status":"done","queue_duration_s":0.49528,"scheduling_latency_s":0.495299,"redis_calls":7,"redis_duration_s":0.002105,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001592,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000513,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037798,"completed_at":"2026-09-08T18:43:02.301Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098gitlab # [ 421.913249] gitaly[906]: time="2026-09-08T18:43:02.037Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T18:43:12.013" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:02.024" grpc.time_ms=13.465 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2099gitlab # [ 421.926530] gitaly[906]: time="2026-09-08T18:43:02.075Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:43:02.075" grpc.time_ms=0.486 pid=906 span.kind=server system=grpc2100gitlab # [ 421.936537] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6916b70dea546f59100e000d","created_at":"2026-09-08T18:43:01.768Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.770Z","job_size_bytes":6,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6916b70dea546f59100e000d: start","job_status":"start","queue_duration_s":0.535199,"scheduling_latency_s":0.535288}2101gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.17 seconds)2102gitlab # [ 421.952452] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.341Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6916b70dea546f59100e000d","created_at":"2026-09-08T18:43:01.768Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.770Z","job_size_bytes":6,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6916b70dea546f59100e000d: done: 0.036044 sec","job_status":"done","queue_duration_s":0.535199,"scheduling_latency_s":0.535288,"redis_calls":7,"redis_duration_s":0.0023290000000000003,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036044,"completed_at":"2026-09-08T18:43:02.341Z","load_balancing_strategy":"primary","db_duration_s":0.000699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2103gitlab # [ 421.993470] nix-daemon[3709]: accepted connection from pid 4710, user nixbot2104gitlab # [ 421.994462] gitaly[906]: time="2026-09-08T18:43:02.099Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T18:43:02.099" grpc.time_ms=0.411 pid=906 span.kind=server system=grpc2105gitlab # [ 422.003940] gitaly[906]: time="2026-09-08T18:43:02.105Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R diskcache=1886283c-acd3-4342-b986-13d2830fa7d2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T18:43:12.031" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:43:02.041" pid=906 ref=refs/keep-around/bb589a2d290709cbf570f8d4a4f14687f88c80e3 remote_ip="2001:db8:1::1" revision=bb589a2d290709cbf570f8d4a4f14687f88c80e3 span.kind=server system=grpc username=root2106gitlab # [ 422.015250] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"eedce070ff0247f693cf2bd9","created_at":"2026-09-08T18:43:01.770Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.772Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-eedce070ff0247f693cf2bd9: start","job_status":"start","queue_duration_s":0.571953,"scheduling_latency_s":0.571985}2107gitlab # [ 422.030170] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.385Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"eedce070ff0247f693cf2bd9","created_at":"2026-09-08T18:43:01.770Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.772Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-eedce070ff0247f693cf2bd9: done: 0.041695 sec","job_status":"done","queue_duration_s":0.571953,"scheduling_latency_s":0.571985,"redis_calls":7,"redis_duration_s":0.00229,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000664,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041695,"completed_at":"2026-09-08T18:43:02.385Z","load_balancing_strategy":"primary","db_duration_s":0.005481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108gitlab # [ 422.071452] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1132109gitlab # [ 422.074139] gitaly[906]: time="2026-09-08T18:43:02.106Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=58 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5419 command.oublock=8 command.real_time_ms=59 command.spawn_token_fork_ms=0 command.system_time_ms=28 command.user_time_ms=30 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T18:43:12.031" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T18:43:02.041" grpc.time_ms=64.204 pid=906 ref=refs/keep-around/bb589a2d290709cbf570f8d4a4f14687f88c80e3 remote_ip="2001:db8:1::1" revision=bb589a2d290709cbf570f8d4a4f14687f88c80e3 span.kind=server system=grpc username=root2110gitlab # [ 422.088376] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.388Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ab9c15e7a3e78a658a5fcf38","created_at":"2026-09-08T18:43:01.832Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.835Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab9c15e7a3e78a658a5fcf38: start","job_status":"start","queue_duration_s":0.552741,"scheduling_latency_s":0.552769}2111gitlab # [ 422.103577] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.429Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ab9c15e7a3e78a658a5fcf38","created_at":"2026-09-08T18:43:01.832Z","trace_propagation_headers":{"sentry-trace":"15f4e4084df645fd894c1bc4f2cf28ff-a8a6d2bd6a024cbc","baggage":"sentry-trace_id=15f4e4084df645fd894c1bc4f2cf28ff,sentry-release=Unknown"},"correlation_id":"01M21598ZYA1KZT9AB3EATG3QZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:01.835Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab9c15e7a3e78a658a5fcf38: done: 0.04119 sec","job_status":"done","queue_duration_s":0.552741,"scheduling_latency_s":0.552769,"redis_calls":8,"redis_duration_s":0.0029100000000000003,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00223,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00068,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04119,"completed_at":"2026-09-08T18:43:02.429Z","load_balancing_strategy":"primary","db_duration_s":0.007002,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2112gitlab # [ 422.144440] gitaly[906]: time="2026-09-08T18:43:02.110Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:32.109" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:02.109" grpc.time_ms=0.492 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113gitlab # [ 422.156518] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.432Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cc70a8d992417c2c7e5e17bb","created_at":"2026-09-08T18:43:02.111Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-4d4fe9cf35d84d79","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.115Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cc70a8d992417c2c7e5e17bb: start","job_status":"start","queue_duration_s":0.316688,"scheduling_latency_s":0.316737}2114gitlab # [ 422.172511] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.500Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cc70a8d992417c2c7e5e17bb","created_at":"2026-09-08T18:43:02.111Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-4d4fe9cf35d84d79","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.115Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cc70a8d992417c2c7e5e17bb: done: 0.067959 sec","job_status":"done","queue_duration_s":0.316688,"scheduling_latency_s":0.316737,"redis_calls":7,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000786,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067959,"completed_at":"2026-09-08T18:43:02.499Z","load_balancing_strategy":"primary","db_duration_s":0.010149,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2115gitlab # [ 422.214058] gitaly[906]: time="2026-09-08T18:43:02.694Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159BAWSW77VTC5H1ZA4D7R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:32.692" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:02.692" grpc.time_ms=1.906 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 422.226276] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.503Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"978e6a8b801622ed84529299","created_at":"2026-09-08T18:43:02.116Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-4d4fe9cf35d84d79","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.119Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-978e6a8b801622ed84529299: start","job_status":"start","queue_duration_s":0.384505,"scheduling_latency_s":0.384543}2117gitlab # [ 422.243215] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.636Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"978e6a8b801622ed84529299","created_at":"2026-09-08T18:43:02.116Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-4d4fe9cf35d84d79","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.119Z","job_size_bytes":65,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-978e6a8b801622ed84529299: done: 0.132829 sec","job_status":"done","queue_duration_s":0.384505,"scheduling_latency_s":0.384543,"redis_calls":10,"redis_duration_s":0.0032059999999999996,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001591,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001615,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.132829,"completed_at":"2026-09-08T18:43:02.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118gitlab # [ 422.287367] gitaly[906]: time="2026-09-08T18:43:02.736Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T18:43:32.435" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:02.735" grpc.time_ms=0.982 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2119gitlab # [ 422.299871] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.638Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e03af5d3b573c50be5dbb196","created_at":"2026-09-08T18:43:02.175Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-4d4fe9cf35d84d79","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.179Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e03af5d3b573c50be5dbb196: start","job_status":"start","queue_duration_s":0.458934,"scheduling_latency_s":0.458965}2120gitlab # [ 422.315410] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.803Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e03af5d3b573c50be5dbb196","created_at":"2026-09-08T18:43:02.175Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-4d4fe9cf35d84d79","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.179Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e03af5d3b573c50be5dbb196: done: 0.16527 sec","job_status":"done","queue_duration_s":0.458934,"scheduling_latency_s":0.458965,"gitaly_calls":2,"gitaly_duration_s":0.006925,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007687,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000946,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001384,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000247,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001773,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000706,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000649,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126228,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1229421,"duration_s":0.16527,"completed_at":"2026-09-08T18:43:02.803Z","load_balancing_strategy":"primary","db_duration_s":0.031254,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2121gitlab # [ 422.366094] gitaly[906]: time="2026-09-08T18:43:02.788Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159AKC5VVVRTV0D4F2752R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:32.487" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:02.787" grpc.time_ms=0.841 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2122gitlab # [ 422.378499] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.806Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"cff0f7f9a913c1deb657eb3c","created_at":"2026-09-08T18:43:02.716Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-187674b8601f4501","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.718Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-cff0f7f9a913c1deb657eb3c: start","job_status":"start","queue_duration_s":0.087468,"scheduling_latency_s":0.087492}2123gitlab # [ 422.394244] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:02.849Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"cff0f7f9a913c1deb657eb3c","created_at":"2026-09-08T18:43:02.716Z","trace_propagation_headers":{"sentry-trace":"ae13cde48bd343e8b8e1b40d0cf601db-187674b8601f4501","baggage":"sentry-trace_id=ae13cde48bd343e8b8e1b40d0cf601db,sentry-release=Unknown"},"correlation_id":"01M2159AKC5VVVRTV0D4F2752R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:02.718Z","job_size_bytes":3,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-cff0f7f9a913c1deb657eb3c: done: 0.04337 sec","job_status":"done","queue_duration_s":0.087468,"scheduling_latency_s":0.087492,"redis_calls":7,"redis_duration_s":0.002097,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04337,"completed_at":"2026-09-08T18:43:02.849Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2125gitlab # [ 423.172693] gitaly[906]: time="2026-09-08T18:43:03.941Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159CGA3NJCCE2WB31ZECNQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:33.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:03.940" grpc.time_ms=1.203 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2126gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.42 seconds)2127gitlab # [ 423.395858] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 3462128gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2129gitlab # [ 424.542698] gitaly[906]: time="2026-09-08T18:43:05.311Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159DW2TPEDA6XFP16ZFB44 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:35.308" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:05.309" grpc.time_ms=1.521 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2130gitlab # [ 424.588712] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1382131gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.20 seconds)2132gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2133gitlab # [ 425.756534] gitaly[906]: time="2026-09-08T18:43:06.525Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159F1HWE9XA2WX16W2QB89 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:36.523" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:06.523" grpc.time_ms=1.265 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.22 seconds)2135gitlab # [ 425.807707] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1592136gitlab # [ 426.082905] nix-daemon[3709]: accepted connection from pid 4746, user nixbot2137gitlab # [ 426.111954] nix-daemon[3709]: accepted connection from pid 4710, user nixbot2138gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2139gitlab # [ 426.976324] gitaly[906]: time="2026-09-08T18:43:07.745Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159G7H740PGPTKZ3VGJNA3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:37.742" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:07.743" grpc.time_ms=1.112 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2140gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.22 seconds)2141gitlab # [ 427.032210] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1662142gitlab # [ 427.206047] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:07.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7b0a2337296af41e2cd710e7","created_at":"2026-09-08T18:43:07.810Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"212e603911e79436c76567df54df4421","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:07.967Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7b0a2337296af41e2cd710e7: start","job_status":"start","queue_duration_s":0.007262,"scheduling_latency_s":0.007319}2143gitlab # [ 427.675747] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:08.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7b0a2337296af41e2cd710e7","created_at":"2026-09-08T18:43:07.810Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"212e603911e79436c76567df54df4421","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:07.967Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7b0a2337296af41e2cd710e7: done: 0.470493 sec","job_status":"done","queue_duration_s":0.007262,"scheduling_latency_s":0.007319,"redis_calls":11,"redis_duration_s":0.054506,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.011116,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.020742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.022648,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.470493,"completed_at":"2026-09-08T18:43:08.444Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.137148,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2144gitlab # [ 427.764270] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:08.464Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6192fb7556acae4d3aac22cb","created_at":"2026-09-08T18:43:08.007Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79595f37db9f316d1d0c49922efea2f6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.045Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6192fb7556acae4d3aac22cb: start","job_status":"start","queue_duration_s":0.419647,"scheduling_latency_s":0.419696}2145gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2146gitlab # [ 428.115727] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:08.884Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6192fb7556acae4d3aac22cb","created_at":"2026-09-08T18:43:08.007Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79595f37db9f316d1d0c49922efea2f6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.045Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6192fb7556acae4d3aac22cb: done: 0.419669 sec","job_status":"done","queue_duration_s":0.419647,"scheduling_latency_s":0.419696,"redis_calls":9,"redis_duration_s":0.036156,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004747,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.020135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.011274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.094,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.094,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.419669,"completed_at":"2026-09-08T18:43:08.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.114141,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2147gitlab # [ 428.212796] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:08.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"61dbcf7b97ac40ff2910b799","created_at":"2026-09-08T18:43:08.068Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f2da4140839ed561446e3b4c82f09da","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.125Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-61dbcf7b97ac40ff2910b799: start","job_status":"start","queue_duration_s":0.78933,"scheduling_latency_s":0.78938}2148gitlab # [ 428.236281] gitaly[906]: time="2026-09-08T18:43:08.956Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159HDYPRR1Y58T9MAAGPVN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:38.053" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:08.954" grpc.time_ms=1.573 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.22 seconds)2150gitlab # [ 428.252995] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1572151gitlab # [ 428.581466] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:09.350Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"61dbcf7b97ac40ff2910b799","created_at":"2026-09-08T18:43:08.068Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f2da4140839ed561446e3b4c82f09da","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.125Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-61dbcf7b97ac40ff2910b799: done: 0.43561 sec","job_status":"done","queue_duration_s":0.78933,"scheduling_latency_s":0.78938,"redis_calls":12,"redis_duration_s":0.069455,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.010001,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01772,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.041734,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.079,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.079,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04952032,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.43561,"completed_at":"2026-09-08T18:43:09.350Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.079276,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2152gitlab # [ 428.674596] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:09.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fee95d6d1bb31f43091a9d09","created_at":"2026-09-08T18:43:08.179Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05fec70dcc536c3acd49d16d5afebf80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.212Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fee95d6d1bb31f43091a9d09: start","job_status":"start","queue_duration_s":1.153472,"scheduling_latency_s":1.153521}2153gitlab # [ 428.892762] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:09.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fee95d6d1bb31f43091a9d09","created_at":"2026-09-08T18:43:08.179Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05fec70dcc536c3acd49d16d5afebf80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.212Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fee95d6d1bb31f43091a9d09: done: 0.295806 sec","job_status":"done","queue_duration_s":1.153472,"scheduling_latency_s":1.153521,"redis_calls":9,"redis_duration_s":0.031258,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.009978,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.017281,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003999,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.295806,"completed_at":"2026-09-08T18:43:09.661Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 428.981352] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:09.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"88a1eb6371309e42d32392ce","created_at":"2026-09-08T18:43:08.240Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b45ad738c306119760c7926de249b4fc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.279Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-88a1eb6371309e42d32392ce: start","job_status":"start","queue_duration_s":1.414539,"scheduling_latency_s":1.414586}2155gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2156gitlab # [ 429.376483] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"88a1eb6371309e42d32392ce","created_at":"2026-09-08T18:43:08.240Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b45ad738c306119760c7926de249b4fc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.279Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-88a1eb6371309e42d32392ce: done: 0.450853 sec","job_status":"done","queue_duration_s":1.414539,"scheduling_latency_s":1.414586,"redis_calls":18,"redis_duration_s":0.22037700000000002,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.009173,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.003252,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.02577,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.182182,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.450853,"completed_at":"2026-09-08T18:43:10.144Z","load_balancing_strategy":"primary","db_duration_s":0.009901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2157gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.22 seconds)2158gitlab # [ 429.480248] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4c4ada6fb325717bc0a14ba0","created_at":"2026-09-08T18:43:08.304Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"701bfa6b31bdfbb7e55951c07ac9a251","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.348Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4c4ada6fb325717bc0a14ba0: start","job_status":"start","queue_duration_s":1.800535,"scheduling_latency_s":1.800583}2159gitlab # [ 429.502263] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1562160gitlab # [ 429.506761] gitaly[906]: time="2026-09-08T18:43:10.192Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159JM7C10ZN13TBZYFP445 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:40.189" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:10.190" grpc.time_ms=2.161 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161gitlab # [ 429.659396] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4c4ada6fb325717bc0a14ba0","created_at":"2026-09-08T18:43:08.304Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"701bfa6b31bdfbb7e55951c07ac9a251","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.348Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4c4ada6fb325717bc0a14ba0: done: 0.27923 sec","job_status":"done","queue_duration_s":1.800535,"scheduling_latency_s":1.800583,"redis_calls":11,"redis_duration_s":0.032848999999999996,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00663,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.011063,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00374754,"exclusive_lock_hold_duration_s":0.00768354,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.27923,"completed_at":"2026-09-08T18:43:10.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162gitlab # [ 429.752459] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"da0776865531ed3298ad057b","created_at":"2026-09-08T18:43:08.398Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e13653b2fa189ef69237f1ebef1a9693","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T18:43:08.451Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-da0776865531ed3298ad057b: start","job_status":"start","queue_duration_s":1.991412,"scheduling_latency_s":1.991459}2163gitlab # [ 429.837440] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"da0776865531ed3298ad057b","created_at":"2026-09-08T18:43:08.398Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e13653b2fa189ef69237f1ebef1a9693","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T18:43:08.451Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-da0776865531ed3298ad057b: done: 0.16391 sec","job_status":"done","queue_duration_s":1.991412,"scheduling_latency_s":1.991459,"redis_calls":9,"redis_duration_s":0.023772,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01225,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005066,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.16391,"completed_at":"2026-09-08T18:43:10.606Z","load_balancing_strategy":"primary","db_duration_s":0.014491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 429.927116] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3bdc0444642e18b2f57b00cc","created_at":"2026-09-08T18:43:08.479Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9546bc77a11731e58144136aa5196787","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T18:43:08.520Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3bdc0444642e18b2f57b00cc: start","job_status":"start","queue_duration_s":2.097141,"scheduling_latency_s":2.097189}2165gitlab # [ 429.944241] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.710Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2166gitlab # [ 429.986981] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3bdc0444642e18b2f57b00cc","created_at":"2026-09-08T18:43:08.479Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9546bc77a11731e58144136aa5196787","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T18:43:08.520Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3bdc0444642e18b2f57b00cc: done: 0.138384 sec","job_status":"done","queue_duration_s":2.097141,"scheduling_latency_s":2.097189,"redis_calls":9,"redis_duration_s":0.025419,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006208,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006374,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.138384,"completed_at":"2026-09-08T18:43:10.755Z","load_balancing_strategy":"primary","db_duration_s":0.001458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2167gitlab # [ 430.080267] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.768Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"29eada118272a54d92581aa3","created_at":"2026-09-08T18:43:08.566Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0a8f7d2dd792c88cb174b871db6c9bf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.605Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-29eada118272a54d92581aa3: start","job_status":"start","queue_duration_s":2.163178,"scheduling_latency_s":2.163229}2168gitlab # [ 430.104426] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.854Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2169gitlab # [ 430.137215] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"29eada118272a54d92581aa3","created_at":"2026-09-08T18:43:08.566Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0a8f7d2dd792c88cb174b871db6c9bf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.605Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-29eada118272a54d92581aa3: done: 0.137505 sec","job_status":"done","queue_duration_s":2.163178,"scheduling_latency_s":2.163229,"redis_calls":9,"redis_duration_s":0.028789,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004469,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.009538,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.137505,"completed_at":"2026-09-08T18:43:10.906Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170gitlab # [ 430.228091] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:10.917Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"49e24a613611491ff7d28e7b","created_at":"2026-09-08T18:43:08.645Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9efc1e0116971c09bd09878505b3c319","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.686Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-49e24a613611491ff7d28e7b: start","job_status":"start","queue_duration_s":2.231131,"scheduling_latency_s":2.231174}2171gitlab # [ 430.367628] nix-daemon[3709]: accepted connection from pid 4710, user nixbot2172gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2173gitlab # [ 430.644825] gitaly[906]: time="2026-09-08T18:43:11.412Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159KTH7ZV6BH9MAAY0ZK1W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:41.410" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:11.411" grpc.time_ms=0.966 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2174gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.20 seconds)2175gitlab # [ 430.685459] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1402176gitlab # [ 430.700318] gitaly[906]: time="2026-09-08T18:43:11.470Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159KQ6M5RE9HN8H9EBD60K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:41.468" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:11.468" grpc.time_ms=1.175 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2177gitlab # [ 430.765368] gitaly[906]: time="2026-09-08T18:43:11.535Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2159KQ6M5RE9HN8H9EBD60K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:21.060" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:11.521" grpc.time_ms=13.927 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2178gitlab # [ 430.819161] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.587Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"49e24a613611491ff7d28e7b","created_at":"2026-09-08T18:43:08.645Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9efc1e0116971c09bd09878505b3c319","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.686Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-49e24a613611491ff7d28e7b: done: 0.669907 sec","job_status":"done","queue_duration_s":2.231131,"scheduling_latency_s":2.231174,"redis_calls":70,"redis_duration_s":0.141726,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.024778,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000981,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.109402,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006565,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.21655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.669907,"completed_at":"2026-09-08T18:43:11.587Z","load_balancing_strategy":"primary","db_duration_s":0.015576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2179gitlab # [ 430.899729] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 4472180gitlab # [ 430.904150] nixbot[3531]: {"ts": "2026-09-08T18:43:11.655Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 \"HTTP/1.1 201 Created\""}2181gitlab # [ 430.908197] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6b72d161f05937177907d14b","created_at":"2026-09-08T18:43:08.726Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8929e11e978336d2ffffaea9d08a6d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.778Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6b72d161f05937177907d14b: start","job_status":"start","queue_duration_s":2.810784,"scheduling_latency_s":2.810817}2182gitlab # [ 430.925993] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6b72d161f05937177907d14b","created_at":"2026-09-08T18:43:08.726Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8929e11e978336d2ffffaea9d08a6d0f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.778Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6b72d161f05937177907d14b: done: 0.037056 sec","job_status":"done","queue_duration_s":2.810784,"scheduling_latency_s":2.810817,"redis_calls":11,"redis_duration_s":0.002715,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001314,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000623,"exclusive_lock_hold_duration_s":0.01055756,"duration_s":0.037056,"completed_at":"2026-09-08T18:43:11.626Z","load_balancing_strategy":"primary","db_duration_s":0.005095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183gitlab # [ 430.969644] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"242ea6f837b9a94557d4dd1e","created_at":"2026-09-08T18:43:08.854Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce046ed0ef3246a62cff29b7dcce1675","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.924Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-242ea6f837b9a94557d4dd1e: start","job_status":"start","queue_duration_s":2.703563,"scheduling_latency_s":2.703587}2184gitlab # [ 430.984063] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.642Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2185gitlab # [ 430.987135] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"242ea6f837b9a94557d4dd1e","created_at":"2026-09-08T18:43:08.854Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce046ed0ef3246a62cff29b7dcce1675","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:08.924Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-242ea6f837b9a94557d4dd1e: done: 0.024135 sec","job_status":"done","queue_duration_s":2.703563,"scheduling_latency_s":2.703587,"redis_calls":9,"redis_duration_s":0.0019979999999999998,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000471,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024135,"completed_at":"2026-09-08T18:43:11.652Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186gitlab # [ 431.029506] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.655Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6f2ebad996777e690d7a494","created_at":"2026-09-08T18:43:08.952Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32563b94b7fdf8c82bbeb8f9463b3f73","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.042Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6f2ebad996777e690d7a494: start","job_status":"start","queue_duration_s":2.613307,"scheduling_latency_s":2.613332}2187gitlab # [ 431.043491] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.704Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6f2ebad996777e690d7a494","created_at":"2026-09-08T18:43:08.952Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32563b94b7fdf8c82bbeb8f9463b3f73","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.042Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6f2ebad996777e690d7a494: done: 0.048636 sec","job_status":"done","queue_duration_s":2.613307,"scheduling_latency_s":2.613332,"redis_calls":9,"redis_duration_s":0.0029259999999999998,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000634,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048636,"completed_at":"2026-09-08T18:43:11.704Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188gitlab # [ 431.085278] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:11.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"610c79ce0fabd5c69642fae0","created_at":"2026-09-08T18:43:09.060Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4ae79fb608a229ce21cc3b641dad711","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.103Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-610c79ce0fabd5c69642fae0: start","job_status":"start","queue_duration_s":2.60302,"scheduling_latency_s":2.603068}2189gitlab # [ 431.100528] gitaly[906]: time="2026-09-08T18:43:11.755Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159M5FZ10MRPY6TXK0AS3A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:41.752" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:11.753" grpc.time_ms=1.566 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2190gitlab # [ 431.113132] gitaly[906]: time="2026-09-08T18:43:11.856Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2159M5FZ10MRPY6TXK0AS3A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:21.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:11.841" grpc.time_ms=14.854 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2191gitlab # [ 431.156271] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2622192gitlab # [ 431.158989] nixbot[3531]: {"ts": "2026-09-08T18:43:11.927Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 \"HTTP/1.1 201 Created\""}2193gitlab # [ 431.226993] gitaly[906]: time="2026-09-08T18:43:11.997Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159ME0F39DS5MJKJ1K34ZW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:41.095" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:11.996" grpc.time_ms=0.803 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2194gitlab # [ 431.376260] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"610c79ce0fabd5c69642fae0","created_at":"2026-09-08T18:43:09.060Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4ae79fb608a229ce21cc3b641dad711","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.103Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-610c79ce0fabd5c69642fae0: done: 0.439136 sec","job_status":"done","queue_duration_s":2.60302,"scheduling_latency_s":2.603068,"redis_calls":564,"redis_duration_s":0.092016,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.090401,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.389168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.439136,"completed_at":"2026-09-08T18:43:12.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000838,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 431.428999] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7a3d8987626311d6122bb256","created_at":"2026-09-08T18:43:09.137Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"561670c70d0333893cb0221a81abd2db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.183Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7a3d8987626311d6122bb256: start","job_status":"start","queue_duration_s":2.964235,"scheduling_latency_s":2.964265}2196gitlab # [ 431.443853] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.161Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2197gitlab # [ 431.447796] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7a3d8987626311d6122bb256","created_at":"2026-09-08T18:43:09.137Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"561670c70d0333893cb0221a81abd2db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.183Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7a3d8987626311d6122bb256: done: 0.022088 sec","job_status":"done","queue_duration_s":2.964235,"scheduling_latency_s":2.964265,"redis_calls":9,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022088,"completed_at":"2026-09-08T18:43:12.169Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2198gitlab # [ 431.490597] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"15cab4a7e26a9442896b3b14","created_at":"2026-09-08T18:43:09.249Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa02a99596ccee10a40ab311238d618e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T18:43:09.292Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-15cab4a7e26a9442896b3b14: start","job_status":"start","queue_duration_s":2.878991,"scheduling_latency_s":2.879018}2199gitlab # [ 431.504101] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.184Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2200gitlab # [ 431.507699] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"15cab4a7e26a9442896b3b14","created_at":"2026-09-08T18:43:09.249Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa02a99596ccee10a40ab311238d618e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T18:43:09.292Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-15cab4a7e26a9442896b3b14: done: 0.021111 sec","job_status":"done","queue_duration_s":2.878991,"scheduling_latency_s":2.879018,"redis_calls":9,"redis_duration_s":0.001629,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000392,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021111,"completed_at":"2026-09-08T18:43:12.192Z","load_balancing_strategy":"primary","db_duration_s":0.000807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201gitlab # [ 431.550537] gitaly[906]: time="2026-09-08T18:43:12.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2159ME0F39DS5MJKJ1K34ZW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T18:43:22.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T18:43:12.238" grpc.time_ms=8.206 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2202gitlab # [ 431.563946] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c4b96d32c12f4b83f7da6761","created_at":"2026-09-08T18:43:09.327Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"777c126081331a6915c1e586da637274","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.397Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c4b96d32c12f4b83f7da6761: start","job_status":"start","queue_duration_s":2.796686,"scheduling_latency_s":2.796714}2203gitlab # [ 431.577807] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c4b96d32c12f4b83f7da6761","created_at":"2026-09-08T18:43:09.327Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"777c126081331a6915c1e586da637274","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.397Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c4b96d32c12f4b83f7da6761: done: 0.05593 sec","job_status":"done","queue_duration_s":2.796686,"scheduling_latency_s":2.796714,"redis_calls":9,"redis_duration_s":0.002362,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001448,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.05593,"completed_at":"2026-09-08T18:43:12.249Z","load_balancing_strategy":"primary","db_duration_s":0.021033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2204gitlab # [ 431.619661] nixbot[3531]: {"ts": "2026-09-08T18:43:12.331Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 \"HTTP/1.1 201 Created\""}2205gitlab # [ 431.622633] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb589a2d290709cbf570f8d4a4f14687f88c80e3 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 3922206gitlab # [ 431.625213] sidekiq-cluster[3399]: {"severity":"INFO","time":"2026-09-08T18:43:12.253Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5344fb48a7bc96ecbb8dc12e","created_at":"2026-09-08T18:43:09.435Z","trace_propagation_headers":{"sentry-trace":"d7027c2226ad41b698ba7b905a71cc7d-3d959576c9a14612","baggage":"sentry-trace_id=d7027c2226ad41b698ba7b905a71cc7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"691883a0bbefd7a99e536ba05d53c91f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T18:43:09.469Z","job_size_bytes":2,"pid":3399,"sidekiq_tid":"m8mn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5344fb48a7bc96ecbb8dc12e: start","job_status":"start","queue_duration_s":2.784038,"scheduling_latency_s":2.784093}2207gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses2208gitlab # [ 431.812486] gitaly[906]: time="2026-09-08T18:43:12.581Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2159MZNDDSRQ5WG897RP4TX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T18:43:42.580" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T18:43:12.580" grpc.time_ms=0.791 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2209gitlab # [ 431.862230] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/08:18:43:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1302210gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bb589a2d290709cbf570f8d4a4f14687f88c80e3/statuses, in 0.18 seconds)2211(finished: subtest: eval and build post commit statuses, in 10.06 seconds)2212(finished: run the VM test script, in 502.29 seconds)2213test script finished in 502.33s2214cleanup2215kill QemuMachine (pid 46)2216gitlab # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2217(finished: cleanup, in 1.53 seconds)2218additionally exposed symbols:2219 gitlab,2220 vlan1,2221 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2222[]2223[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-08T18:41:36.780Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2224[]2225[]2226[]2227[]2228[]2229[]2230[]2231[]2232[]2233[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-08T18:43:01.863208Z', 'started_at': '2026-09-08T18:43:02.211303Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2234[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2235[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2236[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2237[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2238[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2239[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2240[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2241[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2242[{'id': 5, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-09-08T18:43:02.167Z', 'started_at': None, 'finished_at': '2026-09-08T18:43:11.590Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'bb589a2d290709cbf570f8d4a4f14687f88c80e3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-08T18:43:11.891Z', 'started_at': None, 'finished_at': '2026-09-08T18:43:12.279Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]