vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #492
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.ySWOYIEhqT', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: 706059ac-4873-43f8-a269-982f0281bbed16gitlab # Superblock backups stored on blocks:17gitlab # 32768, 98304, 163840, 22937618(finished: start all VMs, in 201.77 seconds)19gitlab # 20gitlab: waiting for unit setup-gitlab.service21gitlab # Allocating group tables: 0/8 done22gitlab: waiting for the VM to finish booting23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/53y2gwj6193fv6k7m82brwwyw6dvmxmc-nixos-system-gitlab-test/init regInfo=/nix/store/5myv8mr1xgv0nvzihjmh7580zwxzri5n-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/5myv8mr1xgv0nvzihjmh7580zwxzri5n-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000042] arm-pv: using stolen time PV108gitlab # [ 0.000433] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000640] Console: colour dummy device 80x25110gitlab # [ 0.000648] printk: legacy console [tty0] enabled111gitlab # [ 0.000849] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000856] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000945] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.001108] landlock: Up and running.115gitlab # [ 0.001112] Yama: becoming mindful.116gitlab # [ 0.001574] LSM support for eBPF active117gitlab # [ 0.002349] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.002468] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.004969] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.006131] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.006141] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.006434] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.007672] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.007783] EFI services will not be available.125gitlab # [ 0.008011] smp: Bringing up secondary CPUs ...126gitlab # [ 0.009150] Detected PIPT I-cache on CPU1127gitlab # [ 0.009220] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.009298] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.009379] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.012683] Detected PIPT I-cache on CPU2131gitlab # [ 0.012765] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.012850] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.012931] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.014158] Detected PIPT I-cache on CPU3135gitlab # [ 0.014236] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.014311] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.014389] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.015555] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.015583] SMP: Total of 4 processors activated.140gitlab # [ 0.015585] CPU: All CPU(s) started at EL1141gitlab # [ 0.015594] CPU features: detected: Branch Target Identification142gitlab # [ 0.015598] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.015604] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.015615] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.015620] CPU features: detected: Common not Private translations146gitlab # [ 0.015623] CPU features: detected: CRC32 instructions147gitlab # [ 0.015626] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.015633] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.015644] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.015652] CPU features: detected: E0PD151gitlab # [ 0.015657] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.015660] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.015663] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.015666] CPU features: detected: Fine Grained Traps155gitlab # [ 0.015669] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.015673] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.015676] CPU features: detected: LSE atomic instructions158gitlab # [ 0.015679] CPU features: detected: Privileged Access Never159gitlab # [ 0.015682] CPU features: detected: PMUv3160gitlab # [ 0.015684] CPU features: detected: RAS Extension Support161gitlab # [ 0.015687] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.015689] CPU features: detected: Random Number Generator163gitlab # [ 0.015691] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.015694] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.015697] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.015700] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.015791] alternatives: applying system-wide alternatives168gitlab # [ 0.019441] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.019919] Memory: 5963768K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.024112] devtmpfs: initialized171gitlab # [ 0.031128] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.031687] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.031967] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.031975] 0 pages in range for non-PLT usage175gitlab # [ 0.031976] 508288 pages in range for PLT usage176gitlab # [ 0.032133] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.033116] DMI not present or invalid.178gitlab # [ 0.037749] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.042157] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.042453] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.044939] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.044981] audit: initializing netlink subsys (disabled)183gitlab # [ 0.046259] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.047338] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.047345] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.047353] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.047355] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.047358] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.047479] cpuidle: using governor ladder190gitlab # [ 0.047558] cpuidle: using governor menu191gitlab # [ 0.047837] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.048138] ASID allocator initialised with 65536 entries193gitlab # [ 0.049812] Serial: AMBA PL011 UART driver194gitlab # [ 0.056867] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.057452] printk: console [ttyAMA0] enabled196gitlab # [ 0.064408] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.064425] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.064430] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.064434] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.064438] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.064442] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.064447] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.064451] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.071547] fbcon: Taking over console205gitlab # [ 0.071580] ACPI: Interpreter disabled.206gitlab # [ 0.073368] iommu: Default domain type: Translated207gitlab # [ 0.073380] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.091172] SCSI subsystem initialized209gitlab # [ 0.093870] usbcore: registered new interface driver usbfs210gitlab # [ 0.093938] usbcore: registered new interface driver hub211gitlab # [ 0.093969] usbcore: registered new device driver usb212gitlab # [ 0.094686] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.094694] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.094707] PTP clock support registered215gitlab # [ 0.094773] EDAC MC: Ver: 3.0.0216gitlab # [ 0.095295] scmi_core: SCMI protocol bus registered217gitlab # [ 0.097162] FPGA manager framework218gitlab # [ 0.099596] vgaarb: loaded219gitlab # [ 0.100713] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.106572] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.106616] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.107330] netfs: FS-Cache loaded223gitlab # [ 0.107485] pnp: PnP ACPI: disabled224gitlab # [ 0.122672] NET: Registered PF_INET protocol family225gitlab # [ 0.126608] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.180028] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.180120] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.180203] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.181772] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.182450] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.182628] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.182737] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.182906] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.183132] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.183155] NET: Registered PF_XDP protocol family236gitlab # [ 0.183172] PCI: CLS 0 bytes, default 64237gitlab # [ 0.183418] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.193336] kvm [1]: HYP mode not available239gitlab # [ 0.263879] Initialise system trusted keyrings240gitlab # [ 0.264864] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.265573] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.267302] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.282082] Key type asymmetric registered244gitlab # [ 0.282107] Asymmetric key parser 'x509' registered245gitlab # [ 0.282537] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.283081] io scheduler mq-deadline registered247gitlab # [ 0.283098] io scheduler kyber registered248gitlab # [ 0.296601] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.301200] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.302329] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.302362] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.302386] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.302398] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.302436] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.302464] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.302579] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.302588] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.302592] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.302596] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.302599] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.302709] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.303389] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.303913] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.303933] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.303975] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.303991] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.304681] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.304988] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.305004] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.305037] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.305716] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.306004] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.306020] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.306054] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.306712] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.306985] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.307000] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.307034] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.308488] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.308780] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.308795] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.308829] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.309468] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.309702] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.309717] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.309759] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.310419] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.310696] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.310710] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.310743] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.310757] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.311430] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.311705] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.311736] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.312365] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.312636] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.312668] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.313213] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.313449] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.313793] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.314268] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.314284] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.314315] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.314922] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.315177] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.315191] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.315222] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.316906] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.316914] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.316918] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.316978] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.317032] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.317090] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.317148] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.317214] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.317266] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.317324] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.317382] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.317442] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.317511] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.317563] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.317584] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.317604] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.317626] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.317647] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.317668] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.317695] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.317717] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.317743] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.317765] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.317797] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.317824] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.317844] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.317880] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.317901] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.317922] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.317950] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.317974] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.317995] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.318027] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.318048] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.318079] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.318085] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.318089] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.319665] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.338001] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.343437] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.346040] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.353861] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.356524] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.361094] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.367603] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.369791] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.375507] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.381448] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.384448] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.390401] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.392790] msm_serial: driver initialized359gitlab # [ 0.392992] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.393046] STM32 USART driver initialized361gitlab # [ 0.421996] loop: module loaded362gitlab # [ 0.422503] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.425459] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.431588] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.433997] virtio_blk virtio5: [vdb] 12395656 512-byte logical blocks (6.35 GB/5.91 GiB)366gitlab # [ 0.437642] megasas: 07.734.00.00-rc1367gitlab # [ 0.438454] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.452802] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.452831] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.454704] Using buffer write method371gitlab # [ 0.454776] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.457322] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.457365] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.460066] Using buffer write method375gitlab # [ 0.460099] Concatenating MTD devices:376gitlab # [ 0.460102] (0): "0.flash"377gitlab # [ 0.460105] (1): "0.flash"378gitlab # [ 0.460107] into device "0.flash"379gitlab # [ 0.924791] Freeing initrd memory: 27184K380gitlab # [ 0.940571] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.948164] thunder_xcv, ver 1.0382gitlab # [ 0.948337] thunder_bgx, ver 1.0383gitlab # [ 0.948542] nicpf, ver 1.0384gitlab # [ 0.949725] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.949737] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.949774] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.949782] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.949823] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.949829] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.949908] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.949914] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.950170] sky2: driver version 1.30393gitlab # [ 0.953124] usbcore: registered new interface driver usb-storage394gitlab # [ 0.953362] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.953387] usbserial: USB Serial support registered for generic396gitlab # [ 0.954063] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.954135] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.954507] hv_vmbus: registering driver hyperv_keyboard399gitlab # [ 0.954649] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000400gitlab # [ 0.956439] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.956479] rtc-pl031 9010000.pl031: setting system clock to 2026-11-10T10:59:01 UTC (1794308341)402gitlab # [ 0.957056] i2c_dev: i2c /dev entries driver403gitlab # [ 0.962117] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.962127] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.962669] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.963425] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.966531] hid: raw HID events driver (C) Jiri Kosina408gitlab # [ 0.967386] usbcore: registered new interface driver usbhid409gitlab # [ 0.967395] usbhid: USB HID core driver410gitlab # [ 0.968770] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00411gitlab # [ 0.970192] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available412gitlab # [ 0.970424] hub 1-0:1.0: USB hub found413gitlab # [ 0.970478] hub 1-0:1.0: 6 ports detected414gitlab # [ 0.973095] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.973448] NET: Registered PF_INET6 protocol family416gitlab # [ 0.974927] Segment Routing with IPv6417gitlab # [ 0.974965] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.975119] NET: Registered PF_PACKET protocol family419gitlab # [ 0.975473] 9pnet: Installing 9P2000 support420gitlab # [ 0.978614] Key type dns_resolver registered421gitlab # [ 0.990412] registered taskstats version 1422gitlab # [ 0.990882] Loading compiled-in X.509 certificates423gitlab # [ 1.010237] Demotion targets for Node 0: null424gitlab # [ 1.010482] Key type .fscrypt registered425gitlab # [ 1.010489] Key type fscrypt-provisioning registered426gitlab # [ 1.010683] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 1.010732] ima: Allocated hash algorithm: sha1428gitlab # [ 1.010771] ima: No architecture policies found429gitlab # [ 1.012011] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 1.042794] clk: Disabling unused clocks431gitlab # [ 1.042851] PM: genpd: Disabling unused power domains432gitlab # [ 1.048900] Freeing unused kernel memory: 4736K433gitlab # [ 1.049313] Run /init as init process434gitlab # [ 1.091837] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 1.224804] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.379928] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.426391] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.426520] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.426590] systemd[1]: Detected architecture arm64.440gitlab # [ 1.426598] systemd[1]: Running in initrd.441gitlab # [ 1.427275] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.427516] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.506063] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.628810] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.725356] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.788373] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2447gitlab # [ 1.788762] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0448gitlab # [ 1.832209] systemd[1]: Queued start job for default target Initrd Default Target.449gitlab # [ 1.871597] systemd[1]: Created slice Slice /system/modprobe.450gitlab # [ 1.872340] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.451gitlab # [ 1.872431] systemd[1]: Expecting device /dev/disk/by-label/nix-store...452gitlab # [ 1.872474] systemd[1]: Expecting device /dev/disk/by-label/nixos...453gitlab # [ 1.872586] systemd[1]: Reached target Path Units.454gitlab # [ 1.872614] systemd[1]: Reached target Slice Units.455gitlab # [ 1.872637] systemd[1]: Reached target Swaps.456gitlab # [ 1.872662] systemd[1]: Reached target Timer Units.457gitlab # [ 1.872960] systemd[1]: Listening on D-Bus System Message Bus Socket.458gitlab # [ 1.873181] systemd[1]: Listening on Journal Socket (/dev/log).459gitlab # [ 1.873453] systemd[1]: Listening on Journal Sockets.460gitlab # [ 1.873759] systemd[1]: Listening on udev Control Socket.461gitlab # [ 1.873968] systemd[1]: Listening on udev Kernel Socket.462gitlab # [ 1.874007] systemd[1]: Reached target Socket Units.463gitlab # [ 1.886352] systemd[1]: Starting Create List of Static Device Nodes...464gitlab # [ 1.888652] systemd[1]: Starting Load Kernel Module 9pnet_virtio...465gitlab # [ 1.888782] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs466gitlab # [ 1.896564] systemd[1]: Mounting Kernel Configuration File System...467gitlab # [ 1.924986] systemd[1]: Starting Journal Service...468gitlab # [ 1.939539] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.939870] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.943408] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.948790] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.952505] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.959815] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.962190] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.989205] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...476gitlab # [ 1.992654] systemd-journald[96]: Collecting audit messages is disabled.477gitlab # [ 2.016230] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.478gitlab # [ 2.016706] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev479gitlab # [ 2.026720] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0480gitlab # [ 2.027120] [drm] features: -virgl +edid -resource_blob -host_visible481gitlab # [ 2.027133] [drm] features: -context_init482gitlab # [ 2.028875] [drm] number of scanouts: 1483gitlab # [ 2.028896] [drm] number of cap sets: 0484gitlab # [ 2.034444] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic485gitlab # [ 2.034490] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0486gitlab # [ 2.041006] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.487gitlab # [ 2.048956] systemd[1]: Starting Create Static Device Nodes in /dev...488gitlab # [ 2.049154] Console: switching to colour frame buffer device 160x50489gitlab # [ 2.066266] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device490gitlab # [ 2.103077] systemd[1]: Finished Load Kernel Modules.491gitlab # [ 2.118639] systemd[1]: Starting Apply Kernel Variables...492gitlab # [ 2.119142] systemd-modules-load[97]: Using 4 probe threads493gitlab # [ 2.142411] systemd[1]: Started Journal Service.494gitlab # [ 2.133676] systemd-modules-load[97]: Module 'virtio_balloon' is built in495gitlab # [ 2.758300] systemd-modules-load[97]: Module 'virtio_console' is built in496gitlab # [ 2.760229] systemd-modules-load[97]: Module 'virtio_rng' is built in497gitlab # [ 2.761969] systemd-modules-load[97]: Inserted module 'dm_mod'498gitlab # [ 2.763516] systemd-modules-load[97]: Inserted module 'virtio_gpu'499gitlab # [ 2.765390] systemd[1]: Finished Create Static Device Nodes in /dev.500gitlab # [ 2.767132] systemd[1]: Reached target Preparation for Local File Systems.501gitlab # [ 2.769015] systemd[1]: Reached target Local File Systems.502gitlab # [ 2.770567] systemd[1]: Starting Create System Files and Directories...503gitlab # [ 2.772363] systemd[1]: Starting Rule-based Manager for Device Events and Files...504gitlab # [ 2.774194] systemd[1]: Finished Apply Kernel Variables.505gitlab # [ 2.775500] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 2.777508] systemd-udevd[109]: Using default interface naming scheme 'v261'.507gitlab # [ 2.779133] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 2.780813] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 2.782226] systemd-vconsole-setup[122]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 2.784740] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.877527] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.879056] systemd[1]: Reached target System Initialization.513gitlab # [ 2.880435] systemd[1]: Reached target Basic System.514gitlab # [ 3.244551] (udev-worker)[142]: Network interface NamePolicy= disabled on kernel command line.515gitlab # [ 3.248871] (udev-worker)[135]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.516gitlab # [ 3.251472] (udev-worker)[135]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 3.285635] systemd[1]: Found device /dev/disk/by-label/nix-store.518gitlab # [ 3.316922] systemd[1]: Found device /dev/disk/by-label/nixos.519gitlab # [ 3.318468] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 3.320202] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 3.366981] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 3.371087] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 3.376042] systemd[1]: Mounting /sysroot...524gitlab # [ 3.469578] EXT4-fs (vda): mounted filesystem 706059ac-4873-43f8-a269-982f0281bbed r/w with ordered data mode. Quota mode: none.525gitlab # [ 3.465021] systemd[1]: Mounted /sysroot.526gitlab # [ 3.466317] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 3.468862] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 3.474396] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 3.476520] systemd[1]: Mounting /sysroot/run...530gitlab # [ 3.497637] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 3.498705] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 3.508857] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 3.518965] systemd[1]: Mounted /sysroot/nix/.rw-store.534gitlab # [ 3.524289] systemd[1]: Starting rw-sysroot-nix-store.service...535gitlab # [ 3.531865] systemd-sysroot-fstab-check[165]: /sysroot should be mounted in the initrd, will request daemon-reload.536gitlab # [ 3.535645] systemd[1]: Mounted /sysroot/tmp/shared.537gitlab # [ 3.538094] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.538gitlab # [ 3.540447] systemd[1]: Finished rw-sysroot-nix-store.service.539gitlab # [ 3.544531] systemd[1]: Mounted /sysroot/run.540gitlab # [ 3.545450] systemd[1]: Mounted /sysroot/tmp/xchg.541gitlab # [ 3.547712] systemd[1]: Reload requested from client PID 165 ('systemd-sysroot') (unit initrd-parse-etc.service)...542gitlab # [ 3.549677] systemd[1]: Reloading...543gitlab # [ 3.730571] erofs (device vdb): mounted with root inode @ nid 128.544gitlab # [ 3.788168] systemd[1]: Reloading finished in 239 ms.545gitlab # [ 3.843411] systemd-sysroot-fstab-check[165]: Requesting initrd-fs.target/start/replace...546gitlab # [ 3.845464] systemd[1]: Mounted /sysroot/nix/.ro-store.547gitlab # [ 3.850080] systemd-sysroot-fstab-check[165]: Requesting swap.target/start/replace...548gitlab # [ 3.868094] systemd[1]: Starting rw-sysroot-nix-store.service...549gitlab # [ 3.873513] systemd[1]: initrd-parse-etc.service: Deactivated successfully.550gitlab # [ 3.874963] systemd[1]: Finished Mountpoints Configured in the Real Root.551gitlab # [ 3.876201] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.552gitlab # [ 3.956878] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.553gitlab # [ 3.958712] systemd[1]: Finished rw-sysroot-nix-store.service.554gitlab # [ 3.968896] systemd[1]: Mounting /sysroot/nix/store...555gitlab # [ 4.086684] (udev-worker)[115]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory556gitlab # [ 4.091208] (udev-worker)[115]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory557gitlab # [ 4.095916] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.558gitlab # [ 4.097716] systemd[1]: Stopped Virtual Console Setup.559gitlab # [ 4.098773] systemd[1]: Stopping Virtual Console Setup...560gitlab # [ 4.099803] systemd[1]: Starting Virtual Console Setup...561gitlab # [ 4.131345] systemd[1]: Mounted /sysroot/nix/store.562gitlab # [ 4.207601] systemd[1]: Reached target Initrd File Systems.563gitlab # [ 4.209596] systemd[1]: Starting Find NixOS closure...564gitlab # [ 4.212616] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...565gitlab # [ 4.220533] systemd-vconsole-setup[199]: Configuration of first virtual console was skipped, ignoring remaining ones.566gitlab # [ 4.233159] systemd[1]: Finished Virtual Console Setup.567gitlab # [ 4.236078] systemd[1]: sysroot-run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully.568gitlab # [ 4.247839] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.569gitlab # [ 4.266169] systemd[1]: Finished Find NixOS closure.570gitlab # [ 4.267591] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 4.269179] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 4.310189] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 4.311625] systemd[1]: Stopped target Basic System.574gitlab # [ 4.312884] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 4.314300] systemd[1]: Stopped target Path Units.576gitlab # [ 4.315459] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 4.317297] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 4.319037] systemd[1]: Stopped target Slice Units.579gitlab # [ 4.320203] systemd[1]: Stopped target Socket Units.580gitlab # [ 4.321378] systemd[1]: Stopped target System Initialization.581gitlab # [ 4.322689] systemd[1]: Stopped target Swaps.582gitlab # [ 4.323754] systemd[1]: Stopped target Timer Units.583gitlab # [ 4.324977] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 4.327305] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 4.328815] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 4.335059] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 4.336408] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 4.337921] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 4.339559] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 4.342130] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 4.343399] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 4.348359] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 4.349667] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 4.351912] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 4.354171] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 4.355768] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 4.357515] systemd[1]: Stopped target Local File Systems.598gitlab # [ 4.358730] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 4.360166] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 4.361686] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 4.362951] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 4.364695] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 4.366422] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 4.367552] systemd[1]: initrd-cleanup.service: Deactivated successfully.605gitlab # [ 4.369039] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.606gitlab # [ 4.370387] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.607gitlab # [ 4.371893] systemd[1]: Finished Load Kernel Module 9pnet_virtio.608gitlab # [ 4.373312] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 4.374792] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 4.376003] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 4.377478] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 4.379193] systemd[1]: systemd-udevd.service: Consumed 3.110s CPU time over 2.128s wall clock time, 30.7M memory peak.613gitlab # [ 4.381491] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 4.382904] systemd[1]: Closed udev Control Socket.615gitlab # [ 4.383910] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 4.385100] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 4.386627] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 4.387824] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 4.389462] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 4.390753] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 4.392034] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 4.431803] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 4.433729] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 4.434964] systemd[1]: Reached target Switch Root.625gitlab # [ 4.436159] systemd[1]: Starting NixOS Activation...626gitlab # [ 4.634196] initrd-nixos-activation-start[223]: booting system configuration /nix/store/53y2gwj6193fv6k7m82brwwyw6dvmxmc-nixos-system-gitlab-test627gitlab # [ 4.689014] initrd-nixos-activation-start[223]: running activation script...628gitlab # [ 4.961887] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.629gitlab # [ 5.362011] initrd-nixos-activation-start[246]: setting up /etc...630gitlab # [ 5.537628] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.631gitlab # [ 5.539556] systemd[1]: Finished NixOS Activation.632gitlab # [ 5.540849] systemd[1]: Starting Switch Root...633gitlab # [ 5.593719] systemd[1]: Switching root.634gitlab # [ 5.752746] systemd-journald[96]: Received SIGTERM from PID 1 (systemd).635gitlab # [ 6.490047] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)636gitlab # [ 6.490920] systemd[1]: Detected virtualization qemu.637gitlab # [ 6.491707] systemd[1]: Detected architecture arm64.638gitlab # [ 6.495113] systemd[1]: Detected first boot.639gitlab # [ 6.496235] systemd[1]: Initializing machine ID from random generator.640gitlab # [ 6.766748] systemd[1]: bpf-restrict-fs: LSM BPF program attached641gitlab # [ 6.912658] systemd[1]: Applying preset policy.642gitlab # [ 7.282803] systemd[1]: Populated /etc with preset unit settings.643gitlab # [ 7.687354] systemd[1]: initrd-switch-root.service: Deactivated successfully.644gitlab # [ 7.688090] systemd[1]: Stopped initrd-switch-root.service.645gitlab # [ 7.691104] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.646gitlab # [ 7.695051] systemd[1]: Created slice Slice /system/getty.647gitlab # [ 7.697430] systemd[1]: Created slice GitLab DevOps Platform Slice.648gitlab # [ 7.699873] systemd[1]: Created slice User and Session Slice.649gitlab # [ 7.700266] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.650gitlab # [ 7.702371] systemd[1]: Started Forward Password Requests to Wall Directory Watch.651gitlab # [ 7.702963] systemd[1]: Expecting device /dev/hvc0...652gitlab # [ 7.703367] systemd[1]: Expecting device /dev/ttyAMA0...653gitlab # [ 7.703771] systemd[1]: Reached target Local Encrypted Volumes.654gitlab # [ 7.704196] systemd[1]: Stopped target initrd-fs.target.655gitlab # [ 7.704582] systemd[1]: Stopped target initrd-root-fs.target.656gitlab # [ 7.705220] systemd[1]: Stopped target initrd-switch-root.target.657gitlab # [ 7.705924] systemd[1]: Reached target Virtual Machines and Containers.658gitlab # [ 7.706591] systemd[1]: Reached target Path Units.659gitlab # [ 7.707235] systemd[1]: Reached target Remote File Systems.660gitlab # [ 7.707856] systemd[1]: Reached target Slice Units.661gitlab # [ 7.708475] systemd[1]: Reached target Swaps.662gitlab # [ 7.710898] systemd[1]: Listening on Query the User Interactively for a Password.663gitlab # [ 7.714209] systemd[1]: Listening on Process Core Dump Socket.664gitlab # [ 7.716095] systemd[1]: Listening on Credential Encryption/Decryption.665gitlab # [ 7.718243] systemd[1]: Listening on Factory Reset Management.666gitlab # [ 7.719061] systemd[1]: Listening on Hostname Service Socket.667gitlab # [ 7.726201] systemd[1]: Starting Journal Log Access Socket...668gitlab # [ 7.728326] systemd[1]: Listening on Journal Audit Socket.669gitlab # [ 7.731661] systemd[1]: Listening on Console Output Muting Service Socket.670gitlab # [ 7.733225] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.671gitlab # [ 7.734547] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os672gitlab # [ 7.735217] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki673gitlab # [ 7.739099] systemd[1]: Listening on Disk Repartitioning Service Socket.674gitlab # [ 7.740167] systemd[1]: Listening on udev Control Socket.675gitlab # [ 7.741131] systemd[1]: Listening on udev Varlink Socket.676gitlab # [ 7.764940] systemd[1]: Mounting Huge Pages File System...677gitlab # [ 7.768658] systemd[1]: Mounting POSIX Message Queue File System...678gitlab # [ 7.774249] systemd[1]: Mounting Kernel Debug File System...679gitlab # [ 7.778816] systemd[1]: Mounting Kernel Trace File System...680gitlab # [ 7.785228] systemd[1]: Starting Create List of Static Device Nodes...681gitlab # [ 7.796472] systemd[1]: Starting Load Kernel Module 9pnet_virtio...682gitlab # [ 7.799076] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs683gitlab # [ 7.805603] systemd[1]: Mounting Kernel Configuration File System...684gitlab # [ 7.808468] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm685gitlab # [ 7.810389] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore686gitlab # [ 7.818771] systemd[1]: Starting Load Kernel Module fuse...687gitlab # [ 7.819564] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67688gitlab # [ 7.833058] systemd[1]: Starting Journal Service...689gitlab # [ 7.839522] systemd[1]: Starting Load Kernel Modules...690gitlab # [ 7.848936] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...691gitlab # [ 7.855787] fuse: init (API version 7.45)692gitlab # [ 7.857776] systemd[1]: Starting Remount Root and Kernel File Systems...693gitlab # [ 7.859160] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os694gitlab # [ 7.865391] systemd[1]: Starting Coldplug All udev Devices...695gitlab # [ 7.870428] systemd[1]: Listening on Journal Log Access Socket.696gitlab # [ 7.871323] systemd[1]: Mounted Huge Pages File System.697gitlab # [ 7.872130] systemd[1]: Mounted POSIX Message Queue File System.698gitlab # [ 7.872920] systemd[1]: Mounted Kernel Debug File System.699gitlab # [ 7.873730] systemd[1]: Mounted Kernel Trace File System.700gitlab # [ 7.874920] systemd[1]: Finished Create List of Static Device Nodes.701gitlab # [ 7.876165] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.702gitlab # [ 7.879478] systemd[1]: Finished Load Kernel Module 9pnet_virtio.703gitlab # [ 7.880389] systemd[1]: Mounted Kernel Configuration File System.704gitlab # [ 7.889167] systemd[1]: modprobe@fuse.service: Deactivated successfully.705gitlab # [ 7.893766] systemd[1]: Finished Load Kernel Module fuse.706gitlab # [ 7.896867] systemd-journald[317]: Collecting audit messages is enabled.707gitlab # [ 7.900972] systemd[1]: Mounting FUSE Control File System...708gitlab # [ 7.905004] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...709gitlab # [ 7.907546] EXT4-fs (vda): re-mounted 706059ac-4873-43f8-a269-982f0281bbed.710gitlab # [ 7.913180] systemd[1]: Finished Load Kernel Modules.711gitlab # [ 7.911047] systemd[1]: Queued start job for default target Multi-User System.712gitlab # [ 7.912458] systemd[1]: Unnecessary job was removed for /dev/vdb.713gitlab # [ 7.913589] systemd[1]: systemd-journald.service: Deactivated successfully.714gitlab # [ 7.914975] systemd-modules-load[318]: Using 3 probe threads715gitlab # [ 7.916318] systemd-modules-load[318]: Module 'atkbd' is built in716gitlab # [ 7.920460] systemd-modules-load[318]: Module 'loop' is built in717gitlab # [ 7.921626] systemd-modules-load[318]: Inserted module 'tls'718gitlab # [ 7.932181] systemd[1]: Starting Firewall...719gitlab # [ 7.936917] systemd[1]: Starting Apply Kernel Variables...720gitlab # [ 7.939250] systemd[1]: Started Journal Service.721gitlab # [ 7.935931] systemd[1]: Finished Remount Root and Kernel File Systems.722gitlab # [ 7.937802] systemd[1]: Mounted FUSE Control File System.723gitlab # [ 7.939572] systemd-oomd[319]: No swap; memory pressure usage will be degraded724gitlab # [ 7.942919] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.725gitlab # [ 7.948823] systemd[1]: Listening on Disk Image Download Service Socket.726gitlab # [ 7.959903] systemd[1]: Starting Flush Journal to Persistent Storage...727gitlab # [ 7.961357] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore728gitlab # [ 7.963318] systemd[1]: Starting Load/Save OS Random Seed...729gitlab # [ 7.967161] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os730gitlab # [ 7.972192] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.731gitlab # [ 7.983978] systemd-journald[317]: Received client request to flush runtime journal.732gitlab # [ 8.016876] systemd[1]: Starting Create Static Device Nodes in /dev...733gitlab # [ 8.020325] systemd[1]: Finished Load/Save OS Random Seed.734gitlab # [ 8.021483] systemd[1]: Finished Apply Kernel Variables.735gitlab # [ 8.022863] systemd[1]: Reached target First Boot Complete.736gitlab # [ 8.023938] systemd[1]: Finished Create Static Device Nodes in /dev.737gitlab # [ 8.025227] systemd[1]: Reached target Preparation for Local File Systems.738gitlab # [ 8.026252] systemd[1]: Starting Rule-based Manager for Device Events and Files...739gitlab # [ 8.027340] systemd[1]: Finished Flush Journal to Persistent Storage.740gitlab # [ 8.101231] systemd-udevd[353]: Using default interface naming scheme 'v261'.741gitlab # [ 8.151666] systemd[1]: Started Rule-based Manager for Device Events and Files.742gitlab # [ 8.526969] systemd[1]: Finished Firewall.743gitlab # [ 8.588517] systemd[1]: Finished Coldplug All udev Devices.744gitlab # [ 8.666960] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs745gitlab # [ 8.691373] systemd[1]: Mounting /run/wrappers...746gitlab # [ 8.764205] systemd[1]: Mounted /run/wrappers.747gitlab # [ 8.766425] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.748gitlab # [ 8.772324] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.749gitlab # [ 8.774014] systemd[1]: Reached target Local File Systems.750gitlab # [ 8.775055] systemd[1]: Listening on Boot Loader Control Service Socket.751gitlab # [ 8.779523] systemd[1]: Starting register-nix-paths.service...752gitlab # [ 8.782664] (udev-worker)[472]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.753gitlab # [ 8.788415] systemd[1]: Starting Create SUID/SGID Wrappers...754gitlab # [ 8.789752] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.755gitlab # [ 8.793022] (udev-worker)[470]: Network interface NamePolicy= disabled on kernel command line.756gitlab # [ 8.796370] (udev-worker)[472]: Network interface NamePolicy= disabled on kernel command line.757gitlab # [ 8.800445] systemd[1]: Starting Save Transient machine-id to Disk...758gitlab # [ 8.804416] systemd[1]: Starting Create System Files and Directories...759gitlab # [ 8.851572] systemd[1]: Condition check resulted in Virtio network device being skipped.760gitlab # [ 8.855176] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore761gitlab # [ 8.865719] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.762gitlab # [ 8.867604] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67763gitlab # [ 8.872194] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore764gitlab # [ 8.874824] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os765gitlab # [ 8.877533] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os766gitlab # [ 8.879819] systemd[1]: Finished Create System Files and Directories.767gitlab # [ 8.883118] systemd[1]: Starting Rebuild Journal Catalog...768gitlab # [ 8.885753] systemd[1]: Starting Record System Boot/Shutdown in UTMP...769gitlab # [ 8.892575] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.770gitlab # [ 8.900682] systemd[1]: Finished Save Transient machine-id to Disk.771gitlab # [ 8.922561] systemd[1]: Finished Record System Boot/Shutdown in UTMP.772gitlab # [ 8.964754] systemd[1]: Finished Rebuild Journal Catalog.773gitlab # [ 8.970178] systemd[1]: Starting Update is Completed...774gitlab # [ 8.980971] mousedev: PS/2 mouse device common for all mice775gitlab # [ 9.001655] systemd[1]: Finished Update is Completed.776gitlab # [ 9.129602] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.777gitlab # [ 9.132658] systemd[1]: Finished Create SUID/SGID Wrappers.778gitlab # [ 9.133816] systemd[1]: suid-sgid-wrappers.service: Consumed 198ms CPU time over 342ms wall clock time, 6.3M memory peak, 1.9M read from disk.779gitlab # [ 9.807564] systemd[1]: Finished register-nix-paths.service.780gitlab # [ 9.820944] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3781gitlab # [ 9.815074] systemd[1]: Reached target System Initialization.782gitlab # [ 9.816338] systemd[1]: Started Discard unused filesystem blocks once a week.783gitlab # [ 9.817468] systemd[1]: Started logrotate.timer.784gitlab # [ 9.818221] systemd[1]: Started Daily Cleanup of Temporary Directories.785gitlab # [ 9.819211] systemd[1]: Reached target Timer Units.786gitlab # [ 9.825251] systemd[1]: Listening on D-Bus System Message Bus Socket.787gitlab # [ 9.826737] systemd[1]: Listening on Nix Daemon Socket.788gitlab # [ 9.874719] systemd[1]: Starting nixbot CI service socket...789gitlab # [ 9.877648] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.790gitlab # [ 9.882839] systemd[1]: Listening on nixbot CI service socket.791gitlab # [ 9.918156] systemd[1]: Reached target Socket Units.792gitlab # [ 9.924178] systemd[1]: Starting D-Bus System Message Bus...793gitlab # [ 9.928793] systemd[1]: Starting Virtual Console Setup...794gitlab # [ 10.015876] dbus-broker-launch[567]: Looking up NSS user entry for 'systemd-timesync'...795gitlab # [ 10.018218] dbus-broker-launch[567]: NSS returned no entry for 'systemd-timesync'796gitlab # [ 10.020524] dbus-broker-launch[567]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"797gitlab # [ 10.024983] systemd-vconsole-setup[568]: Configuration of first virtual console was skipped, ignoring remaining ones.798gitlab # [ 10.027897] systemd[1]: Finished Virtual Console Setup.799gitlab # [ 10.029426] systemd[1]: Started D-Bus System Message Bus.800gitlab # [ 10.045965] dbus-broker-launch[567]: Ready801gitlab # [ 10.047191] systemd[1]: Reached target Basic System.802gitlab # [ 10.051980] systemd[1]: Started backdoor.service.803gitlab # [ 10.058064] systemd[1]: Starting gitlab-config.service...804gitlab # [ 10.060432] systemd[1]: Starting Import lastlog data into lastlog2 database...805gitlab # [ 10.088588] systemd[1]: Starting Logrotate configuration check...806gitlab # [ 10.090910] systemd[1]: Starting Name Service Cache Daemon (nsncd)...807gitlab # [ 10.100877] systemd[1]: Starting Post-Boot Actions...808gitlab # [ 10.110964] systemd[1]: Started Reset console on configuration changes.809gitlab # [ 10.120186] systemd[1]: Starting resolvconf update...810gitlab # connecting to host...811gitlab # [ 10.128430] logrotate[575]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.812gitlab # [ 10.137932] logrotate[575]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf813gitlab # [ 10.143528] logrotate[575]: note: 'monthly' overrides previously specified 'weekly'814gitlab # [ 10.145496] logrotate[575]: Reading state from file: /var/lib/logrotate.status815gitlab # [ 10.147305] logrotate[575]: state file /var/lib/logrotate.status does not exist816gitlab # [ 10.148879] logrotate[575]: Allocating hash table for state file, size 64 entries817gitlab # [ 10.150397] logrotate[575]: Handling 4 logs818gitlab # [ 10.151241] logrotate[575]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed819gitlab # [ 10.154756] logrotate[575]: considering log /var/log/btmp820gitlab # [ 10.155812] logrotate[575]: Creating new state821gitlab # [ 10.156817] logrotate[575]: Now: 2026-11-10 10:59822gitlab # [ 10.157739] logrotate[575]: Last rotated at 2026-11-10 10:00823gitlab # [ 10.158797] logrotate[575]: log does not need rotating (log has already been rotated)824gitlab # [ 10.160315] logrotate[575]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed825gitlab # [ 10.163268] logrotate[575]: considering log /var/log/wtmp826gitlab # [ 10.164222] logrotate[575]: Creating new state827gitlab # [ 10.165023] logrotate[575]: Now: 2026-11-10 10:59828gitlab # [ 10.166158] logrotate[575]: Last rotated at 2026-11-10 10:00829gitlab # [ 10.167227] logrotate[575]: log does not need rotating (log has already been rotated)830gitlab: Guest shell says: b'Spawning backdoor root shell...\n'831gitlab # [ 10.169221] logrotate[575]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed832gitlab # [ 10.171364] logrotate[575]: switching euid from 0 to 165 and egid from 0 to 165 (pid 575)833gitlab # [ 10.172769] logrotate[575]: considering log /var/gitlab/state/log/*.log834gitlab # [ 10.173753] logrotate[575]: log /var/gitlab/state/log/*.log does not exist -- skipping835gitlab: connected to guest root shell836gitlab # [ 10.175052] logrotate[575]: Creating new state837gitlab: (connecting took 10.71 seconds)838gitlab # [ 10.175750] logrotate[575]: switching euid from 165 to 0 and egid from 165 to 0 (pid 575)839gitlab # [ 10.178135] logrotate[575]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed840gitlab: (finished: waiting for the VM to finish booting, in 10.71 seconds)841gitlab # [ 10.180162] logrotate[575]: switching euid from 0 to 60 and egid from 0 to 60 (pid 575)842gitlab # [ 10.181314] logrotate[575]: considering log /var/log/nginx/*.log843gitlab # [ 10.182539] logrotate[575]: log /var/log/nginx/*.log does not exist -- skipping844gitlab # [ 10.183700] logrotate[575]: Creating new state845gitlab # [ 10.184849] logrotate[575]: not running postrotate script, since no logs were rotated846gitlab # [ 10.186182] logrotate[575]: switching euid from 60 to 0 and egid from 60 to 0 (pid 575)847gitlab # [ 10.187630] systemd[1]: Finished Logrotate configuration check.848gitlab # [ 10.189293] nsncd[576]: Nov 10 10:59:10.738 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"849gitlab # [ 10.191600] systemd[1]: Finished Post-Boot Actions.850gitlab # [ 10.193650] systemd[1]: Finished Import lastlog data into lastlog2 database.851gitlab # [ 10.195016] systemd[1]: Started Name Service Cache Daemon (nsncd).852gitlab # [ 10.197387] systemd[1]: Reached target Host and Network Name Lookups.853gitlab # [ 10.198824] systemd[1]: Reached target User and Group Name Lookups.854gitlab # [ 10.199807] systemd[1]: Starting User Login Management...855gitlab # [ 10.271868] systemd[1]: Stopped target Host and Network Name Lookups.856gitlab # [ 10.273440] systemd[1]: Stopping Host and Network Name Lookups...857gitlab # [ 10.274454] systemd[1]: Stopped target User and Group Name Lookups.858gitlab # [ 10.275626] systemd[1]: Stopping User and Group Name Lookups...859gitlab # [ 10.276818] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...860gitlab # [ 10.277791] systemd[1]: nscd.service: Deactivated successfully.861gitlab # [ 10.278637] systemd[1]: Stopped Name Service Cache Daemon (nsncd).862gitlab # [ 10.279617] systemd[1]: nscd.service: Consumed 43ms CPU time over 184ms wall clock time, 7.6M memory peak, 5.8M read from disk.863gitlab # [ 10.286469] systemd-logind[607]: New seat seat0.864gitlab # [ 10.287458] systemd[1]: Starting Name Service Cache Daemon (nsncd)...865gitlab # [ 10.296124] systemd-logind[607]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)866gitlab # [ 10.297722] systemd-logind[607]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)867gitlab # [ 10.299053] systemd-logind[607]: Watching system buttons on /dev/input/event0 (gpio-keys)868gitlab # [ 10.300213] systemd[1]: Started User Login Management.869gitlab # [ 10.314101] systemd[1]: Starting linger-users.service...870gitlab # [ 10.330805] nsncd[665]: Nov 10 10:59:10.882 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"871gitlab # [ 10.333429] systemd[1]: Started Name Service Cache Daemon (nsncd).872gitlab # [ 10.334355] systemd[1]: Reached target Host and Network Name Lookups.873gitlab # [ 10.335354] systemd[1]: Reached target User and Group Name Lookups.874gitlab # [ 10.342701] systemd[1]: Finished resolvconf update.875gitlab # [ 10.343623] systemd[1]: linger-users.service: Deactivated successfully.876gitlab # [ 10.344748] systemd[1]: Finished linger-users.service.877gitlab # [ 10.347474] systemd[1]: Reached target Preparation for Network.878gitlab # [ 10.356138] systemd[1]: Starting DHCP Client...879gitlab # [ 10.360401] systemd[1]: Starting Address configuration of eth1...880gitlab # [ 10.376274] systemd[1]: Starting Extra networking commands....881gitlab # [ 10.419743] network-addresses-eth1-start[693]: adding address 192.168.1.1/24... done882gitlab # [ 10.427290] network-addresses-eth1-start[693]: adding address 2001:db8:1::1/64... done883gitlab # [ 10.437348] systemd[1]: Finished Address configuration of eth1.884gitlab # [ 10.487318] systemd[1]: Finished Extra networking commands..885gitlab # [ 10.489060] dhcpcd[704]: dhcpcd-10.3.2 starting886gitlab # [ 10.490027] systemd[1]: Reached target Network.887gitlab # [ 10.496222] systemd[1]: Starting gitlab-workhorse.service...888gitlab # [ 10.497907] dhcpcd[758]: dev: loaded udev889gitlab # [ 10.498605] systemd[1]: Starting Nginx Web Server...890gitlab # [ 10.502232] systemd[1]: Starting PostgreSQL Server...891gitlab # [ 10.514373] systemd[1]: Starting Redis Server - redis-gitlab...892gitlab # [ 10.520879] systemd[1]: Starting Permit User Sessions...893gitlab # [ 10.535062] 8021q: 802.1Q VLAN Support v1.8894gitlab # [ 10.535521] 8021q: adding VLAN 0 to HW filter on device eth1895gitlab # [ 10.565960] systemd[1]: Finished Permit User Sessions.896gitlab # [ 10.573901] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.897gitlab # [ 10.580635] systemd[1]: Started Getty on tty1.898gitlab # [ 10.581526] systemd[1]: Reached target Login Prompts.899gitlab # [ 10.604214] cfg80211: Loading compiled-in X.509 certificates for regulatory database900gitlab # [ 10.619660] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'901gitlab # [ 10.620213] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'902gitlab # [ 10.625644] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2903gitlab # [ 10.626046] cfg80211: failed to load regulatory.db904gitlab # [ 10.662537] dhcpcd[758]: eth0: waiting for carrier[ 10.669058] 8021q: adding VLAN 0 to HW filter on device eth0905gitlab # 906gitlab # [ 10.665598] dhcpcd[758]: eth0: carrier acquired907gitlab # [ 10.669215] dhcpcd[758]: DUID 00:01:00:01:32:85:b5:7f:52:54:00:12:34:56908gitlab # [ 10.670379] dhcpcd[758]: eth0: IAID 00:12:34:56909gitlab # [ 10.671099] dhcpcd[758]: eth0: adding address fe80::5054:ff:fe12:3456910gitlab # [ 10.749374] nginx-pre-start[784]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok911gitlab # [ 10.752502] nginx-pre-start[784]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful912gitlab # [ 10.761735] systemd[1]: Started Nginx Web Server.913gitlab # [ 10.797613] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.914gitlab # [ 10.800279] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo915gitlab # [ 10.801940] postgresql-pre-start[792]: The files belonging to this database system will be owned by user "postgres".916gitlab # [ 10.803427] postgresql-pre-start[792]: This user must also own the server process.917gitlab # [ 10.804603] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started918gitlab # [ 10.806811] redis-server[788]: Configuration loaded919gitlab # [ 10.807784] systemd[1]: Started Redis Server - redis-gitlab.920gitlab # [ 10.809172] postgresql-pre-start[792]: The database cluster will be initialized with locale "en_US.UTF-8".921gitlab # [ 10.810686] postgresql-pre-start[792]: The default database encoding has accordingly been set to "UTF8".922gitlab # [ 10.812218] postgresql-pre-start[792]: The default text search configuration will be set to "english".923gitlab # [ 10.813782] postgresql-pre-start[792]: Data page checksums are enabled.924gitlab # [ 10.814999] postgresql-pre-start[792]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok925gitlab # [ 10.816548] postgresql-pre-start[792]: creating subdirectories ... ok926gitlab # [ 10.817551] postgresql-pre-start[792]: selecting dynamic shared memory implementation ... posix927gitlab # [ 10.818864] redis[788]: monotonic clock: ARM CNTVCT @ 1000 ticks/us928gitlab # [ 10.820116] redis[788]: Running mode=standalone, port=0.929gitlab # [ 10.821081] redis[788]: Server initialized930gitlab # [ 10.821905] redis[788]: Ready to accept connections unix931gitlab # [ 10.823018] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.932gitlab # [ 10.884146] postgresql-pre-start[792]: selecting default "max_connections" ... 100933gitlab # [ 10.965928] postgresql-pre-start[792]: selecting default "shared_buffers" ... 128MB934gitlab # [ 11.517017] postgresql-pre-start[792]: selecting default time zone ... UTC935gitlab # [ 11.519350] postgresql-pre-start[792]: creating configuration files ... ok936gitlab # [ 11.678354] systemd[1]: Started gitlab-workhorse.service.937gitlab # [ 11.847119] postgresql-pre-start[792]: running bootstrap script ... ok938gitlab # [ 12.148766] gitlab-workhorse[829]: time="2026-11-10T10:59:12Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4939gitlab # [ 12.150766] gitlab-workhorse[829]: time="2026-11-10T10:59:12Z" level=info msg="Using redis/go-redis"940gitlab # [ 12.153765] gitlab-workhorse[829]: time="2026-11-10T10:59:12Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix941gitlab # [ 12.435813] dhcpcd[758]: eth0: soliciting a DHCP lease942gitlab # [ 12.462715] dhcpcd[758]: eth0: offered 10.0.2.15 from 10.0.2.2943gitlab # [ 12.484456] dhcpcd[758]: eth0: probing address 10.0.2.15/24944gitlab # [ 12.906131] postgresql-pre-start[792]: performing post-bootstrap initialization ... ok945gitlab # [ 12.951174] dhcpcd[758]: eth0: soliciting an IPv6 router946gitlab # [ 12.958341] dhcpcd[758]: eth0: Router Advertisement from fe80::2947gitlab # [ 12.959994] dhcpcd[758]: eth0: adding address fec0::5054:ff:fe12:3456/64948gitlab # [ 12.961312] dhcpcd[758]: eth0: adding route to fec0::/64949gitlab # [ 12.962621] dhcpcd[758]: eth0: adding default route via fe80::2950gitlab # [ 13.177852] postgresql-pre-start[792]: syncing data to disk ... ok951gitlab # [ 13.179453] postgresql-pre-start[792]: initdb: warning: enabling "trust" authentication for local connections952gitlab # [ 13.181745] postgresql-pre-start[792]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.953gitlab # [ 13.185297] postgresql-pre-start[792]: Success. You can now start the database server using:954gitlab # [ 13.187285] postgresql-pre-start[792]: pg_ctl -D /var/lib/postgresql/18 -l logfile start955gitlab # [ 13.326619] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit956gitlab # [ 13.329933] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432957gitlab # [ 13.331853] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432958gitlab # [ 13.333749] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"959gitlab # [ 13.347770] postgres[863]: [863] LOG: database system was shut down at 2026-11-10 10:59:13 GMT960gitlab # [ 13.356506] postgres[854]: [854] LOG: database system is ready to accept connections961gitlab # [ 13.360472] systemd[1]: Started PostgreSQL Server.962gitlab # [ 13.368365] systemd[1]: Starting PostgreSQL Setup Scripts...963gitlab # [ 13.777216] postgresql-setup-start[883]: CREATE DATABASE964gitlab # [ 13.850936] postgresql-setup-start[893]: CREATE ROLE965gitlab # [ 13.872171] postgresql-setup-start[896]: ALTER DATABASE966gitlab # [ 13.916251] postgresql-setup-start[901]: CREATE ROLE967gitlab # [ 13.919070] systemd[1]: Finished PostgreSQL Setup Scripts.968gitlab # [ 13.922479] systemd[1]: Reached target PostgreSQL.969gitlab # [ 13.926916] systemd[1]: Starting gitlab-postgresql.service...970gitlab # [ 14.012559] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[888]: mkdir -p /var/gitlab/state/home/.ssh: OK971gitlab # [ 14.015947] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[888]: chmod 700 /var/gitlab/state/home/.ssh: OK972gitlab # [ 14.103340] systemd[1]: Finished gitlab-config.service.973gitlab # [ 14.108683] systemd[1]: Started gitaly.service.974gitlab # [ 14.114369] systemd[1]: Starting gitlab-db-config.service...975gitlab # [ 14.120588] gitlab-postgresql-start[909]: CREATE DATABASE976gitlab # [ 14.174461] gitlab-postgresql-start[928]: CREATE EXTENSION977gitlab # [ 14.313408] gitlab-postgresql-start[931]: CREATE EXTENSION978gitlab # [ 14.318518] systemd[1]: Finished gitlab-postgresql.service.979gitlab # [ 15.101741] gitaly[923]: time="2026-11-10T10:59:15.652Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=923980gitlab # [ 15.106054] gitaly[923]: time="2026-11-10T10:59:15.652Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=923981gitlab # [ 15.107729] gitaly[923]: time="2026-11-10T10:59:15.653Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=923982gitlab # [ 15.116377] gitaly[923]: time="2026-11-10T10:59:15.653Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=923983gitlab # [ 15.118298] gitaly[923]: time="2026-11-10T10:59:15.653Z" level=info msg="Starting Gitaly" pid=923 version=984gitlab # [ 15.576454] gitaly[923]: time="2026-11-10T10:59:16.128Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=474 pid=923985gitlab # [ 15.579436] gitaly[923]: time="2026-11-10T10:59:16.128Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=923986gitlab # [ 15.581252] gitaly[923]: time="2026-11-10T10:59:16.129Z" level=info msg="finished initializing command factory" duration_ms=0 pid=923987gitlab # [ 15.583516] gitaly[923]: time="2026-11-10T10:59:16.129Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-923/git-exec-1941119009.d/git pid=923988gitlab # [ 15.587389] gitaly[923]: time="2026-11-10T10:59:16.139Z" level=info msg="finished detecting git version" duration_ms=10 pid=923989gitlab # [ 15.589421] gitaly[923]: time="2026-11-10T10:59:16.139Z" level=info msg="using Git version" pid=923 version=2.53.g990gitlab # [ 15.591028] gitaly[923]: time="2026-11-10T10:59:16.140Z" level=info msg="clearing disk cache object folder" pid=923 storage=default991gitlab # [ 15.592713] gitaly[923]: time="2026-11-10T10:59:16.140Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1276906847 pid=923 storage=default992gitlab # [ 15.599401] gitaly[923]: time="2026-11-10T10:59:16.141Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1276906847 pid=923 storage=default993gitlab # [ 15.603531] gitaly[923]: time="2026-11-10T10:59:16.141Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=923994gitlab # [ 15.607321] gitaly[923]: time="2026-11-10T10:59:16.141Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=923995gitlab # [ 15.610366] gitaly[923]: time="2026-11-10T10:59:16.141Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1276906847 clear_duration_ms=0 pid=923 storage=default996gitlab # [ 15.613389] gitaly[923]: time="2026-11-10T10:59:16.164Z" level=info msg="counting repositories" pid=923997gitlab # [ 15.614735] gitaly[923]: time="2026-11-10T10:59:16.164Z" level=info msg="starting to count repositories" pid=923 storage_path=/var/gitlab/state/repositories998gitlab # [ 15.616743] gitaly[923]: time="2026-11-10T10:59:16.164Z" level=info msg="finished tempdir cleaner walk" pid=923 storage=default time_ms=0999gitlab # [ 15.618451] gitaly[923]: time="2026-11-10T10:59:16.164Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=923 storage_path=/var/gitlab/state/repositories1000gitlab # [ 15.620668] gitaly[923]: time="2026-11-10T10:59:16.165Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9231001gitlab # [ 15.622380] gitaly[923]: time="2026-11-10T10:59:16.164Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9231002gitlab # [ 15.624612] gitaly[923]: time="2026-11-10T10:59:16.165Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=923 schema=unix1003gitlab # [ 15.626606] gitaly[923]: time="2026-11-10T10:59:16.165Z" level=info msg="listening at address" address=/run/gitaly/gitaly-923/sock.d/intern pid=923 schema=unix1004gitlab # [ 15.628563] gitaly[923]: time="2026-11-10T10:59:16.167Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9231005gitlab # [ 15.631661] gitaly[923]: time="2026-11-10T10:59:16.167Z" level=info msg="Started Gitaly" duration_ms=514 pid=9231006gitlab # [ 15.633321] gitaly[923]: time="2026-11-10T10:59:16.167Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9231007gitlab # [ 15.634987] gitaly[923]: time="2026-11-10T10:59:16.168Z" level=info msg="maintenance: daily scheduled" pid=923 scheduled="2026-11-10 12:00:00 +0000 UTC"1008gitlab # [ 17.243342] dhcpcd[758]: eth0: leased 10.0.2.15 for 86400 seconds1009gitlab # [ 17.243618] dhcpcd[758]: eth0: adding route to 10.0.2.0/241010gitlab # [ 17.243743] dhcpcd[758]: eth0: adding default route via 10.0.2.21011gitlab # [ 17.394925] systemd[1]: Started DHCP Client.1012gitlab # [ 17.398093] systemd[1]: Reached target Network is Online.1013gitlab # [ 17.759032] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[929]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1014gitlab # [ 17.762966] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[929]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1015gitlab # [ 17.831276] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[929]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1016gitlab # [ 20.158482] gitaly[923]: time="2026-11-10T10:59:20.709Z" level=info msg="License database preloaded" duration_ms=5056 pid=9231017gitlab # [ 59.943612] systemd[1]: Starting Logrotate Service...1018gitlab # [ 60.043441] systemd[1]: logrotate.service: Deactivated successfully.1019gitlab # [ 60.045435] systemd[1]: Finished Logrotate Service.1020gitlab # [ 90.362815] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[929]: W, [2026-11-10T11:00:30.913977 #929] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1021gitlab # [ 92.243081] gitlab-workhorse[829]: time="2026-11-10T11:00:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1022gitlab # [ 94.649824] postgres[1723]: [1723] ERROR: relation "feature_gates" does not exist at character 5231023gitlab # [ 94.652368] postgres[1723]: [1723] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1024gitlab # [ 94.654470] postgres[1723]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1025gitlab # [ 94.656673] postgres[1723]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1026gitlab # [ 94.658625] postgres[1723]: attidentity AS identity,1027gitlab # [ 94.659910] postgres[1723]: attgenerated as attgenerated1028gitlab # [ 94.661441] postgres[1723]: FROM pg_attribute a1029gitlab # [ 94.662548] postgres[1723]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1030gitlab # [ 94.664710] postgres[1723]: LEFT JOIN pg_type t ON a.atttypid = t.oid1031gitlab # [ 94.666193] postgres[1723]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1032gitlab # [ 94.668514] postgres[1723]: WHERE a.attrelid = '"feature_gates"'::regclass1033gitlab # [ 94.670047] postgres[1723]: AND a.attnum > 0 AND NOT a.attisdropped1034gitlab # [ 94.671475] postgres[1723]: ORDER BY a.attnum1035gitlab # [ 94.672543] postgres[1723]:1036gitlab # [ 102.244366] gitlab-workhorse[829]: time="2026-11-10T11:00:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1037gitlab # [ 102.447558] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1792]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1038gitlab # [ 102.451150] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1792]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1039gitlab # [ 112.254146] gitlab-workhorse[829]: time="2026-11-10T11:00:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1040gitlab # [ 122.255032] gitlab-workhorse[829]: time="2026-11-10T11:01:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1041gitlab # [ 132.257500] gitlab-workhorse[829]: time="2026-11-10T11:01:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1042gitlab # [ 132.322474] postgres[1949]: [1949] ERROR: relation "table_without_models" does not exist1043gitlab # [ 132.324449] postgres[1949]: [1949] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1044gitlab # [ 142.258774] gitlab-workhorse[829]: time="2026-11-10T11:01:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1045gitlab # [ 149.133647] systemd[1]: Finished gitlab-db-config.service.1046gitlab # [ 149.137445] systemd[1]: Started gitlab-sidekiq.service.1047gitlab # [ 149.142379] systemd[1]: Starting gitlab.service...1048gitlab # [ 149.815024] bundle[2206]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1049gitlab # [ 149.815334] bundle[2206]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1050gitlab # [ 149.820141] bundle[2206]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1051gitlab # [ 152.265611] gitlab-workhorse[829]: time="2026-11-10T11:01:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1052gitlab # [ 157.197454] sidekiq-cluster[2205]: {"severity":"INFO","time":"2026-11-10T11:01:37.703Z","message":"Starting cluster with 1 processes"}1053gitlab # [ 157.220316] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.723Z","pid":2206,"message":"* Enabling systemd notification integration"}1054gitlab # [ 157.223022] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"Puma starting in cluster mode..."}1055gitlab # [ 157.225611] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1056gitlab # [ 157.228170] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1057gitlab # [ 157.231235] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Min threads: 0"}1058gitlab # [ 157.233448] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Max threads: 4"}1059gitlab # [ 157.235501] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Environment: production"}1060gitlab # [ 157.237856] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Master PID: 2206"}1061gitlab # [ 157.240101] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Workers: 2"}1062gitlab # [ 157.242143] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1063gitlab # [ 157.244809] bundle[2206]: {"timestamp":"2026-11-10T11:01:37.772Z","pid":2206,"message":"* Preloading application"}1064gitlab # [ 162.273758] gitlab-workhorse[829]: time="2026-11-10T11:01:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1065gitlab # [ 172.281577] gitlab-workhorse[829]: time="2026-11-10T11:01:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1066gitlab # [ 182.291321] gitlab-workhorse[829]: time="2026-11-10T11:02:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1067gitlab # [ 192.297682] gitlab-workhorse[829]: time="2026-11-10T11:02:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1068gitlab # [ 202.307669] gitlab-workhorse[829]: time="2026-11-10T11:02:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1069gitlab # [ 212.313638] gitlab-workhorse[829]: time="2026-11-10T11:02:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1070gitlab # [ 213.768215] bundle[2206]: W, [2026-11-10T11:02:34.319354 #2206] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1071gitlab # [ 220.115089] sidekiq-cluster[2273]: 2026-11-10T11:02:40.666Z pid=2273 tid=xmx INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2273, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>427786240}1072gitlab # [ 220.912775] sidekiq-cluster[2273]: W, [2026-11-10T11:02:41.464704 #2273] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1073gitlab # [ 220.993720] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:41.545Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1074gitlab # [ 222.323452] gitlab-workhorse[829]: time="2026-11-10T11:02:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1075gitlab # [ 226.659978] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.211Z","message":"GitLab reliable fetch activated!"}1076gitlab # [ 226.660426] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.211Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1077gitlab # [ 226.701569] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.253Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1078gitlab # [ 226.705706] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.256Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1079gitlab # [ 226.709901] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.260Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1080gitlab # [ 226.714017] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.266Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1081gitlab # [ 226.718083] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.269Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1082gitlab # [ 226.721736] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.272Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1083gitlab # [ 226.725421] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.275Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1084gitlab # [ 226.728736] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.278Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1085gitlab # [ 226.732120] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.280Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1086gitlab # [ 226.735426] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.283Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1087gitlab # [ 226.739361] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.285Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1088gitlab # [ 226.743096] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.289Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1089gitlab # [ 226.746566] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.291Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1090gitlab # [ 226.750246] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.294Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1091gitlab # [ 226.753613] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.298Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1092gitlab # [ 226.756781] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.305Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1093gitlab # [ 226.762194] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.314Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1094gitlab # [ 226.767956] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.319Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1095gitlab # [ 226.777311] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.329Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1096gitlab # [ 226.786766] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.338Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1097gitlab # [ 226.796458] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.348Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1098gitlab # [ 226.804409] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.356Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1099gitlab # [ 226.809411] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.361Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1100gitlab # [ 226.814402] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.366Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1101gitlab # [ 226.818953] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.370Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1102gitlab # [ 226.823535] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.375Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1103gitlab # [ 226.827869] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.379Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1104gitlab # [ 226.833060] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.385Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1105gitlab # [ 226.837565] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.389Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1106gitlab # [ 226.841672] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.393Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1107gitlab # [ 226.845109] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.397Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1108gitlab # [ 226.850123] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.402Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1109gitlab # [ 226.853830] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.405Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1110gitlab # [ 226.857770] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.409Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1111gitlab # [ 226.861319] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.413Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1112gitlab # [ 226.865070] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.417Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1113gitlab # [ 226.869052] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.421Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1114gitlab # [ 226.873217] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.425Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1115gitlab # [ 226.876066] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.428Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1116gitlab # [ 226.878789] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.430Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1117gitlab # [ 226.881975] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.433Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1118gitlab # [ 226.884919] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.436Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1119gitlab # [ 226.887845] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.438Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1120gitlab # [ 226.890829] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.441Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1121gitlab # [ 226.893833] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.445Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1122gitlab # [ 226.896749] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.448Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1123gitlab # [ 226.899413] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.450Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1124gitlab # [ 226.902123] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.453Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1125gitlab # [ 226.904625] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.456Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1126gitlab # [ 226.907260] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.459Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1127gitlab # [ 226.910812] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.462Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1128gitlab # [ 226.914139] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.466Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1129gitlab # [ 226.928902] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.480Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1130gitlab # [ 226.933104] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.485Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1131gitlab # [ 226.936235] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.488Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1132gitlab # [ 226.939157] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.491Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1133gitlab # [ 226.942200] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.493Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1134gitlab # [ 226.945355] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.497Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1135gitlab # [ 226.948363] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.499Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1136gitlab # [ 226.951401] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.503Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1137gitlab # [ 226.954194] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.505Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1138gitlab # [ 226.956838] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.507Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1139gitlab # [ 226.959500] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.510Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1140gitlab # [ 226.962294] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.513Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1141gitlab # [ 226.965000] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.515Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1142gitlab # [ 226.967544] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.518Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1143gitlab # [ 226.970031] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.519Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1144gitlab # [ 226.972530] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.521Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1145gitlab # [ 226.975103] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.523Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1146gitlab # [ 226.977835] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.525Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1147gitlab # [ 226.981305] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.527Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1148gitlab # [ 226.983642] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.529Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1149gitlab # [ 226.986163] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.531Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1150gitlab # [ 226.988712] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.533Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1151gitlab # [ 226.991198] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.534Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1152gitlab # [ 226.993819] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.536Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1153gitlab # [ 226.996224] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.538Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1154gitlab # [ 226.998820] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.541Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1155gitlab # [ 227.001253] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.543Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1156gitlab # [ 227.003543] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.544Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1157gitlab # [ 227.006501] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.547Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1158gitlab # [ 227.009202] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.549Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1159gitlab # [ 227.011573] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.552Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1160gitlab # [ 227.013971] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.554Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1161gitlab # [ 227.016381] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.556Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1162gitlab # [ 227.018603] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.558Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1163gitlab # [ 227.022027] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.566Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1164gitlab # [ 227.024521] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.573Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1165gitlab # [ 227.026907] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.577Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1166gitlab # [ 227.029388] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.579Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1167gitlab # [ 227.031779] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.582Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1168gitlab # [ 227.034288] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.584Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1169gitlab # [ 227.036689] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.587Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1170gitlab # [ 227.039460] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.590Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1171gitlab # [ 227.042451] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.594Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1172gitlab # [ 227.045899] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.597Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1173gitlab # [ 227.052205] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.601Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1174gitlab # [ 227.054643] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.604Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1175gitlab # [ 227.063911] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.615Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1176gitlab # [ 227.066641] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.618Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1177gitlab # [ 227.069108] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.620Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1178gitlab # [ 227.123656] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.673Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1179gitlab # [ 227.127956] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.680Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1180gitlab # [ 227.131603] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.683Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1181gitlab # [ 227.136721] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.688Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1182gitlab # [ 227.140578] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.691Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1183gitlab # [ 227.144041] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.693Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1184gitlab # [ 227.147699] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.698Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1185gitlab # [ 227.151541] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:02:47.701Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1186gitlab # [ 232.330400] gitlab-workhorse[829]: time="2026-11-10T11:02:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1187gitlab # [ 242.340191] gitlab-workhorse[829]: time="2026-11-10T11:03:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1188gitlab # [ 252.341919] gitlab-workhorse[829]: time="2026-11-10T11:03:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1189gitlab # [ 262.342683] gitlab-workhorse[829]: time="2026-11-10T11:03:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1190gitlab # [ 272.352754] gitlab-workhorse[829]: time="2026-11-10T11:03:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1191gitlab # [ 280.174804] postgres[3350]: [3350] ERROR: relation "table_without_models" does not exist1192gitlab # [ 280.175093] postgres[3350]: [3350] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1193gitlab # [ 282.361442] gitlab-workhorse[829]: time="2026-11-10T11:03:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1194gitlab # [ 282.848615] postgres[3379]: [3379] ERROR: relation "table_without_models" does not exist1195gitlab # [ 282.850598] postgres[3379]: [3379] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1196gitlab # [ 286.326052] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:46.877Z","message":"Booted Rails 7.2.3.1 application in production environment"}1197gitlab # [ 286.328704] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:46.880Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1198gitlab # [ 286.332465] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:46.883Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1199gitlab # [ 286.334415] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:46.883Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1200gitlab # [ 286.338432] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:46.890Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1201gitlab # [ 289.705951] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:50.257Z","message":"Cleaning working queues"}1202gitlab # [ 289.709449] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:50.259Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2a80422d11778385f4baf318","created_at":"2026-11-10T11:01:29.036Z","trace_propagation_headers":{"sentry-trace":"d6c865045a744d579ca9fbfdffe85f4a-182e06bf80b646b4","baggage":"sentry-trace_id=d6c865045a744d579ca9fbfdffe85f4a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"2f258825edd16c13c10c840d3aae1f22","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T11:01:29.052Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2a80422d11778385f4baf318: start","job_status":"start","queue_duration_s":141.20659,"scheduling_latency_s":141.206662}1203gitlab # [ 290.026169] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:03:50.577Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"2a80422d11778385f4baf318","created_at":"2026-11-10T11:01:29.036Z","trace_propagation_headers":{"sentry-trace":"d6c865045a744d579ca9fbfdffe85f4a-182e06bf80b646b4","baggage":"sentry-trace_id=d6c865045a744d579ca9fbfdffe85f4a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"2f258825edd16c13c10c840d3aae1f22","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T11:01:29.052Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-2a80422d11778385f4baf318: done: 0.318168 sec","job_status":"done","queue_duration_s":141.20659,"scheduling_latency_s":141.206662,"redis_calls":29,"redis_duration_s":0.009998,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006061,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002302,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001635,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.171956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15811174,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.318168,"completed_at":"2026-11-10T11:03:50.577Z","load_balancing_strategy":"primary","db_duration_s":0.077787,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1204gitlab # [ 291.310397] bundle[2206]: {"timestamp":"2026-11-10T11:03:51.861Z","pid":2206,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1205gitlab # [ 291.310665] bundle[2206]: {"timestamp":"2026-11-10T11:03:51.861Z","pid":2206,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1206gitlab # [ 291.310741] bundle[2206]: {"timestamp":"2026-11-10T11:03:51.862Z","pid":2206,"message":"! #\u003cThread:0x0000f85ac45ae4f8 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1207gitlab # [ 291.310807] bundle[2206]: {"timestamp":"2026-11-10T11:03:51.862Z","pid":2206,"message":"! #\u003cThread:0x0000f85ac3e7eb38 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1208gitlab # [ 291.310874] bundle[2206]: {"timestamp":"2026-11-10T11:03:51.862Z","pid":2206,"message":"Use Ctrl-C to stop"}1209gitlab # [ 291.310937] bundle[2206]: {"timestamp":"2026-11-10T11:03:51.862Z","pid":2206,"message":"Sending status to systemd every 1.0 sec"}1210gitlab # [ 292.336762] bundle[2206]: {"timestamp":"2026-11-10T11:03:52.887Z","pid":2206,"message":"- Worker 0 (PID: 3496) booted in 0.98s, phase: 0"}1211gitlab # [ 292.451517] bundle[2206]: {"timestamp":"2026-11-10T11:03:53.003Z","pid":2206,"message":"- Worker 1 (PID: 3500) booted in 1.06s, phase: 0"}1212gitlab # [ 292.460868] systemd[1]: Started gitlab.service.1213gitlab # [ 292.462215] systemd[1]: Reached target Common target for all GitLab services..1214gitlab # [ 292.463856] systemd[1]: Starting setup-gitlab.service...1215gitlab # [ 292.498987] setup-gitlab-start[3536]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1216gitlab # [ 298.782698] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:03:59 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 62321217gitlab # [ 298.787560] setup-gitlab-start[3536]: + TOKEN=nixbot-test-token-of-20-chars1218gitlab # [ 298.789349] setup-gitlab-start[3536]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1219gitlab # [ 298.814935] runuser[3572]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1220gitlab # [ 299.617721] setup-gitlab-start[3573]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1221gitlab # [ 299.621551] setup-gitlab-start[3573]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1222gitlab # [ 299.625010] setup-gitlab-start[3573]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1223gitlab # [ 300.899100] setup-gitlab-start[3573]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1224gitlab # [ 300.899391] setup-gitlab-start[3573]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1225gitlab # [ 300.902737] setup-gitlab-start[3573]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1226gitlab # [ 304.722780] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:05.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2290ce03d201e78ec99c4eee","created_at":"2026-11-10T11:04:05.195Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.269Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2290ce03d201e78ec99c4eee: start","job_status":"start","queue_duration_s":0.004081,"scheduling_latency_s":0.004132}1227gitlab # [ 305.019651] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:05.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2290ce03d201e78ec99c4eee","created_at":"2026-11-10T11:04:05.195Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.269Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2290ce03d201e78ec99c4eee: done: 0.29744 sec","job_status":"done","queue_duration_s":0.004081,"scheduling_latency_s":0.004132,"redis_calls":19,"redis_duration_s":0.038392,"redis_read_bytes":21,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.008841,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.003699,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.015451,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.010401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00360742,"duration_s":0.29744,"completed_at":"2026-11-10T11:04:05.570Z","load_balancing_strategy":"primary","db_duration_s":0.02924,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1228gitlab # [ 305.089814] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:05.576Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a0692441bd9cb2330e30750a","created_at":"2026-11-10T11:04:05.280Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00e848f1b790a3f2edcd0d787df686a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.295Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a0692441bd9cb2330e30750a: start","job_status":"start","queue_duration_s":0.280939,"scheduling_latency_s":0.280983}1229gitlab # [ 306.954751] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:07.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a0692441bd9cb2330e30750a","created_at":"2026-11-10T11:04:05.280Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00e848f1b790a3f2edcd0d787df686a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.295Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a0692441bd9cb2330e30750a: done: 1.927965 sec","job_status":"done","queue_duration_s":0.280939,"scheduling_latency_s":0.280983,"redis_calls":14,"redis_duration_s":0.018252,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001348,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009321,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007583,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.429,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.429,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.582038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00169726,"exclusive_lock_hold_duration_s":1.8283839,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":1.927965,"completed_at":"2026-11-10T11:04:07.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":1.253003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1230gitlab # [ 307.036813] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:07.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"bd49ce33fdc0067136943f66","created_at":"2026-11-10T11:04:05.303Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"481287ced363a613138c1244a692f650","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.326Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-bd49ce33fdc0067136943f66: start","job_status":"start","queue_duration_s":2.2617,"scheduling_latency_s":2.261753}1231gitlab # [ 307.167342] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:07.719Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1232gitlab # [ 307.266779] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:07.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"bd49ce33fdc0067136943f66","created_at":"2026-11-10T11:04:05.303Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"481287ced363a613138c1244a692f650","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.326Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-bd49ce33fdc0067136943f66: done: 0.229507 sec","job_status":"done","queue_duration_s":2.2617,"scheduling_latency_s":2.261753,"redis_calls":10,"redis_duration_s":0.071855,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005143,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.059618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007094,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00482472,"duration_s":0.229507,"completed_at":"2026-11-10T11:04:07.817Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1233gitlab # [ 307.352853] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:07.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"104e7e68348ad1b84f99166f","created_at":"2026-11-10T11:04:05.380Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d013fa013755682965b390c674c4be99","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.394Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-104e7e68348ad1b84f99166f: start","job_status":"start","queue_duration_s":2.50971,"scheduling_latency_s":2.509857}1234gitlab # [ 307.502121] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"104e7e68348ad1b84f99166f","created_at":"2026-11-10T11:04:05.380Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d013fa013755682965b390c674c4be99","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.394Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-104e7e68348ad1b84f99166f: done: 0.149342 sec","job_status":"done","queue_duration_s":2.50971,"scheduling_latency_s":2.509857,"redis_calls":12,"redis_duration_s":0.033745,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.012936,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018593,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002216,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00421614,"duration_s":0.149342,"completed_at":"2026-11-10T11:04:08.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008563,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1235gitlab # [ 307.580238] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fe0f5e659f7fb739123b2dcb","created_at":"2026-11-10T11:04:05.416Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1a4b01c72690dcda11a4c1913f711c4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.456Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fe0f5e659f7fb739123b2dcb: start","job_status":"start","queue_duration_s":2.675595,"scheduling_latency_s":2.675644}1236gitlab # [ 307.812486] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.363Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fe0f5e659f7fb739123b2dcb","created_at":"2026-11-10T11:04:05.416Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1a4b01c72690dcda11a4c1913f711c4","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.456Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fe0f5e659f7fb739123b2dcb: done: 0.231673 sec","job_status":"done","queue_duration_s":2.675595,"scheduling_latency_s":2.675644,"redis_calls":13,"redis_duration_s":0.040191000000000004,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006766,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.021052,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.012373,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.034156320000000004,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.231673,"completed_at":"2026-11-10T11:04:08.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017798,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1237gitlab # [ 307.892810] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.444Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"787a577e9bf368d84b146f26","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-787a577e9bf368d84b146f26: start","job_status":"start","queue_duration_s":2.917459,"scheduling_latency_s":2.917552}1238gitlab # [ 308.041080] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.591Z","message":"No existing merge request to be cleaned up."}1239gitlab # [ 308.120522] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.670Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"787a577e9bf368d84b146f26","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-787a577e9bf368d84b146f26: done: 0.2262 sec","job_status":"done","queue_duration_s":2.917459,"scheduling_latency_s":2.917552,"redis_calls":13,"redis_duration_s":0.034915,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005051,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009792,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.020072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00360386,"duration_s":0.2262,"completed_at":"2026-11-10T11:04:08.670Z","load_balancing_strategy":"primary","db_duration_s":0.035175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1240gitlab # [ 308.183558] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.722Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b1798818b81483cd8f9f2cf7","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b1798818b81483cd8f9f2cf7: start","job_status":"start","queue_duration_s":3.195184,"scheduling_latency_s":3.195243}1241gitlab # [ 308.248189] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.799Z","message":"No existing merge request to be cleaned up."}1242gitlab # [ 308.315350] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.863Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b1798818b81483cd8f9f2cf7","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b1798818b81483cd8f9f2cf7: done: 0.141887 sec","job_status":"done","queue_duration_s":3.195184,"scheduling_latency_s":3.195243,"redis_calls":10,"redis_duration_s":0.025915999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.017393,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.141887,"completed_at":"2026-11-10T11:04:08.863Z","load_balancing_strategy":"primary","db_duration_s":0.030525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 308.369466] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.870Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8ef3c950eb92dfe14b02dd90","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8ef3c950eb92dfe14b02dd90: start","job_status":"start","queue_duration_s":3.343598,"scheduling_latency_s":3.343642}1244gitlab # [ 308.395349] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:08.947Z","message":"No existing merge request to be cleaned up."}1245gitlab # [ 308.464653] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.015Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8ef3c950eb92dfe14b02dd90","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8ef3c950eb92dfe14b02dd90: done: 0.145549 sec","job_status":"done","queue_duration_s":3.343598,"scheduling_latency_s":3.343642,"redis_calls":10,"redis_duration_s":0.032535999999999995,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.013876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.01866,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.145549,"completed_at":"2026-11-10T11:04:09.015Z","load_balancing_strategy":"primary","db_duration_s":0.045316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1246gitlab # [ 308.524594] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.023Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dd0ea50f38f80591c487c4de","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dd0ea50f38f80591c487c4de: start","job_status":"start","queue_duration_s":3.496512,"scheduling_latency_s":3.496561}1247gitlab # [ 308.601702] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.153Z","message":"No existing merge request to be cleaned up."}1248gitlab # [ 308.661467] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.212Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dd0ea50f38f80591c487c4de","created_at":"2026-11-10T11:04:05.439Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-3d1b85eeabb24373","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a49f6c4a3c55c187cefea5e9f8955bb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.526Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dd0ea50f38f80591c487c4de: done: 0.189538 sec","job_status":"done","queue_duration_s":3.496512,"scheduling_latency_s":3.496561,"redis_calls":10,"redis_duration_s":0.027291,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.017311,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.189538,"completed_at":"2026-11-10T11:04:09.212Z","load_balancing_strategy":"primary","db_duration_s":0.030634,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1249gitlab # [ 308.721531] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3b31dabdbfd6d2d38441058b","created_at":"2026-11-10T11:04:05.525Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fdf0e720f72fb6d9fe64675c6a0e5d08","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.547Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3b31dabdbfd6d2d38441058b: start","job_status":"start","queue_duration_s":3.678014,"scheduling_latency_s":3.678066}1250gitlab # [ 308.917986] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3b31dabdbfd6d2d38441058b","created_at":"2026-11-10T11:04:05.525Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fdf0e720f72fb6d9fe64675c6a0e5d08","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.547Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3b31dabdbfd6d2d38441058b: done: 0.242279 sec","job_status":"done","queue_duration_s":3.678014,"scheduling_latency_s":3.678066,"redis_calls":12,"redis_duration_s":0.038825,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007765,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.013201,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0048611,"exclusive_lock_hold_duration_s":0.06610678,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.242279,"completed_at":"2026-11-10T11:04:09.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.062149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1251gitlab # [ 308.991708] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.485Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6cd78abb1484a413ff8ace14","created_at":"2026-11-10T11:04:05.568Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ba9a863ebe2a2443db4205a2251b092","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.621Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6cd78abb1484a413ff8ace14: start","job_status":"start","queue_duration_s":3.863659,"scheduling_latency_s":3.863709}1252gitlab # [ 309.164962] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6cd78abb1484a413ff8ace14","created_at":"2026-11-10T11:04:05.568Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ba9a863ebe2a2443db4205a2251b092","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.621Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6cd78abb1484a413ff8ace14: done: 0.23018 sec","job_status":"done","queue_duration_s":3.863659,"scheduling_latency_s":3.863709,"redis_calls":12,"redis_duration_s":0.030898000000000002,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003885,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009975,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00177772,"exclusive_lock_hold_duration_s":0.025951639999999998,"duration_s":0.23018,"completed_at":"2026-11-10T11:04:09.715Z","load_balancing_strategy":"primary","db_duration_s":0.026898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1253gitlab # [ 309.230127] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:09.731Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c1c039e4a0490f6bf045cf45","created_at":"2026-11-10T11:04:05.708Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"712243d2eac0dfe6f904c0b088f5d900","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.724Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c1c039e4a0490f6bf045cf45: start","job_status":"start","queue_duration_s":4.006758,"scheduling_latency_s":4.006806}1254gitlab # [ 309.844268] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c1c039e4a0490f6bf045cf45","created_at":"2026-11-10T11:04:05.708Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"712243d2eac0dfe6f904c0b088f5d900","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.724Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c1c039e4a0490f6bf045cf45: done: 0.664105 sec","job_status":"done","queue_duration_s":4.006758,"scheduling_latency_s":4.006806,"redis_calls":12,"redis_duration_s":0.032932,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.014798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.474,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.474,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00452916,"exclusive_lock_hold_duration_s":0.025945,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.664105,"completed_at":"2026-11-10T11:04:10.395Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.473656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1255gitlab # [ 309.911081] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"af679bcf1c006caa489e3585","created_at":"2026-11-10T11:04:05.732Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85cd3ae0ba759a390eedcd789868ed03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:04:05.747Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-af679bcf1c006caa489e3585: start","job_status":"start","queue_duration_s":4.653058,"scheduling_latency_s":4.653105}1256gitlab # [ 309.926090] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.439Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1257gitlab # [ 309.929657] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"af679bcf1c006caa489e3585","created_at":"2026-11-10T11:04:05.732Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85cd3ae0ba759a390eedcd789868ed03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:04:05.747Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-af679bcf1c006caa489e3585: done: 0.076356 sec","job_status":"done","queue_duration_s":4.653058,"scheduling_latency_s":4.653105,"redis_calls":10,"redis_duration_s":0.003323,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000723,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000793,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005744,"duration_s":0.076356,"completed_at":"2026-11-10T11:04:10.476Z","load_balancing_strategy":"primary","db_duration_s":0.00151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 309.985628] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"40331af5ef7a9d4c96d5bde3","created_at":"2026-11-10T11:04:05.770Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10eb394a2810cbb505c66805aeee485e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.791Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-40331af5ef7a9d4c96d5bde3: start","job_status":"start","queue_duration_s":4.691146,"scheduling_latency_s":4.691192}1259gitlab # [ 310.031071] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"40331af5ef7a9d4c96d5bde3","created_at":"2026-11-10T11:04:05.770Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10eb394a2810cbb505c66805aeee485e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.791Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-40331af5ef7a9d4c96d5bde3: done: 0.099272 sec","job_status":"done","queue_duration_s":4.691146,"scheduling_latency_s":4.691192,"redis_calls":10,"redis_duration_s":0.01355,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007076,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000832,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055072,"duration_s":0.099272,"completed_at":"2026-11-10T11:04:10.582Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018973,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1260gitlab # [ 310.092453] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.587Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5e2a53fb05e01951076e48cf","created_at":"2026-11-10T11:04:05.804Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16d6ebbd0b404e8e510137cc77fd76e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:04:05.831Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5e2a53fb05e01951076e48cf: start","job_status":"start","queue_duration_s":4.755459,"scheduling_latency_s":4.755501}1261gitlab # [ 310.107338] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.643Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1262gitlab # [ 310.127190] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5e2a53fb05e01951076e48cf","created_at":"2026-11-10T11:04:05.804Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16d6ebbd0b404e8e510137cc77fd76e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:04:05.831Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5e2a53fb05e01951076e48cf: done: 0.091017 sec","job_status":"done","queue_duration_s":4.755459,"scheduling_latency_s":4.755501,"redis_calls":10,"redis_duration_s":0.023254999999999998,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004998,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0035271,"duration_s":0.091017,"completed_at":"2026-11-10T11:04:10.678Z","load_balancing_strategy":"primary","db_duration_s":0.003077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1263gitlab # [ 310.184170] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"22fea4ed7b95316e7182abac","created_at":"2026-11-10T11:04:05.862Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2e1b96df6c3f37e0c5065f1fa896263","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.887Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-22fea4ed7b95316e7182abac: start","job_status":"start","queue_duration_s":4.799025,"scheduling_latency_s":4.799079}1264gitlab # [ 310.208194] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.760Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1265gitlab # [ 310.245852] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"22fea4ed7b95316e7182abac","created_at":"2026-11-10T11:04:05.862Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2e1b96df6c3f37e0c5065f1fa896263","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.887Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-22fea4ed7b95316e7182abac: done: 0.110235 sec","job_status":"done","queue_duration_s":4.799025,"scheduling_latency_s":4.799079,"redis_calls":10,"redis_duration_s":0.035159,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.009528,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.020573,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005058,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00383114,"duration_s":0.110235,"completed_at":"2026-11-10T11:04:10.797Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1266gitlab # [ 310.307516] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.811Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"5725c01ea2de45823a2f48f3","created_at":"2026-11-10T11:04:05.925Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"8fa4fe29cf31ab455d4a2d0c1f4bc4eb","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.950Z","job_size_bytes":3,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-5725c01ea2de45823a2f48f3: start","job_status":"start","queue_duration_s":4.860984,"scheduling_latency_s":4.861035}1267gitlab # [ 310.333004] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.877Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"5725c01ea2de45823a2f48f3","created_at":"2026-11-10T11:04:05.925Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"8fa4fe29cf31ab455d4a2d0c1f4bc4eb","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:05.950Z","job_size_bytes":3,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-5725c01ea2de45823a2f48f3: done: 0.066719 sec","job_status":"done","queue_duration_s":4.860984,"scheduling_latency_s":4.861035,"redis_calls":10,"redis_duration_s":0.004694,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000712,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002043,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064078,"duration_s":0.066719,"completed_at":"2026-11-10T11:04:10.877Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1268gitlab # [ 310.383934] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:10.885Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8ffa9bc7f671a72434c9a46f","created_at":"2026-11-10T11:04:05.978Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f57f79bbdf943fbf5efaf4df6e5f83cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:04:06.001Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8ffa9bc7f671a72434c9a46f: start","job_status":"start","queue_duration_s":4.884495,"scheduling_latency_s":4.884546}1269gitlab # [ 310.509800] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:11.061Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8ffa9bc7f671a72434c9a46f","created_at":"2026-11-10T11:04:05.978Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f57f79bbdf943fbf5efaf4df6e5f83cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:04:06.001Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8ffa9bc7f671a72434c9a46f: done: 0.175357 sec","job_status":"done","queue_duration_s":4.884495,"scheduling_latency_s":4.884546,"redis_calls":10,"redis_duration_s":0.017787,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005381,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0044229,"duration_s":0.175357,"completed_at":"2026-11-10T11:04:11.060Z","load_balancing_strategy":"primary","db_duration_s":0.0661,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 310.574704] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:11.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ecbe76a8df562571e10a6cb9","created_at":"2026-11-10T11:04:06.039Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6be7ad64d3faa242dcc5ea45ffee92b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.120Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ecbe76a8df562571e10a6cb9: start","job_status":"start","queue_duration_s":4.995548,"scheduling_latency_s":4.9956}1271gitlab # [ 310.734094] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:11.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ecbe76a8df562571e10a6cb9","created_at":"2026-11-10T11:04:06.039Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6be7ad64d3faa242dcc5ea45ffee92b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.120Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ecbe76a8df562571e10a6cb9: done: 0.169094 sec","job_status":"done","queue_duration_s":4.995548,"scheduling_latency_s":4.9956,"redis_calls":10,"redis_duration_s":0.028211,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005106,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005071,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00244586,"duration_s":0.169094,"completed_at":"2026-11-10T11:04:11.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002653,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1272gitlab # [ 310.800146] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:11.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5881a3ca16bb841a31788a0a","created_at":"2026-11-10T11:04:06.157Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fd49636987f9ff3bd60b7a8effd8a48","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.209Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5881a3ca16bb841a31788a0a: start","job_status":"start","queue_duration_s":5.082755,"scheduling_latency_s":5.082806}1273gitlab # [ 310.845092] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:11.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5881a3ca16bb841a31788a0a","created_at":"2026-11-10T11:04:06.157Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fd49636987f9ff3bd60b7a8effd8a48","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.209Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5881a3ca16bb841a31788a0a: done: 0.104203 sec","job_status":"done","queue_duration_s":5.082755,"scheduling_latency_s":5.082806,"redis_calls":12,"redis_duration_s":0.0044139999999999995,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000951,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001956,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001507,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00120222,"exclusive_lock_hold_duration_s":0.014442499999999999,"duration_s":0.104203,"completed_at":"2026-11-10T11:04:11.395Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014144,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1274gitlab # [ 310.906410] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:11.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3f6c3352617a4fc9c781a560","created_at":"2026-11-10T11:04:06.233Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e388ef38faab7c9d53a8a344cd6c2678","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.279Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3f6c3352617a4fc9c781a560: start","job_status":"start","queue_duration_s":5.120625,"scheduling_latency_s":5.120676}1275gitlab # [ 311.543821] redis[788]: 10 changes in 300 seconds. Saving...1276gitlab # [ 311.548464] redis[788]: Background saving started by pid 36311277gitlab # [ 311.551896] redis[3631]: BGSAVE done, 391 keys saved, 0 keys skipped, 111897 bytes written.1278gitlab # [ 311.559993] redis[3631]: DB saved on disk1279gitlab # [ 311.561380] redis[3631]: Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB1280gitlab # [ 311.645245] redis[788]: Background saving terminated with success1281gitlab # [ 312.444605] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:12.995Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3f6c3352617a4fc9c781a560","created_at":"2026-11-10T11:04:06.233Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e388ef38faab7c9d53a8a344cd6c2678","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.279Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3f6c3352617a4fc9c781a560: done: 1.594954 sec","job_status":"done","queue_duration_s":5.120625,"scheduling_latency_s":5.120676,"redis_calls":565,"redis_duration_s":0.93013,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000762,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002358,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.92701,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.769467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00104102,"duration_s":1.594954,"completed_at":"2026-11-10T11:04:12.995Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 312.456496] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:12.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"779b272c655c7e1aed4ac967","created_at":"2026-11-10T11:04:06.326Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8ef826d2f16aee1e3671d40cef9617f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.405Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-779b272c655c7e1aed4ac967: start","job_status":"start","queue_duration_s":6.592271,"scheduling_latency_s":6.592308}1283gitlab # [ 312.539320] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.090Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"779b272c655c7e1aed4ac967","created_at":"2026-11-10T11:04:06.326Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8ef826d2f16aee1e3671d40cef9617f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.405Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-779b272c655c7e1aed4ac967: done: 0.092945 sec","job_status":"done","queue_duration_s":6.592271,"scheduling_latency_s":6.592308,"redis_calls":10,"redis_duration_s":0.0037419999999999997,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002345,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0009706,"duration_s":0.092945,"completed_at":"2026-11-10T11:04:13.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1284gitlab # [ 312.620092] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.094Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"702a2579f8c1b25c0c0ff157","created_at":"2026-11-10T11:04:06.440Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0eb340b42402e7cbdc4b4e63470c812c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.486Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-702a2579f8c1b25c0c0ff157: start","job_status":"start","queue_duration_s":6.607459,"scheduling_latency_s":6.607503}1285gitlab # [ 312.635685] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.173Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"702a2579f8c1b25c0c0ff157","created_at":"2026-11-10T11:04:06.440Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0eb340b42402e7cbdc4b4e63470c812c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.486Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-702a2579f8c1b25c0c0ff157: done: 0.079145 sec","job_status":"done","queue_duration_s":6.607459,"scheduling_latency_s":6.607503,"redis_calls":12,"redis_duration_s":0.004383,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001089,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000767,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00069,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053856,"duration_s":0.079145,"completed_at":"2026-11-10T11:04:13.172Z","load_balancing_strategy":"primary","db_duration_s":0.036636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 312.681304] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b14add1f63a26f6a5474a163","created_at":"2026-11-10T11:04:06.535Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5275c55bed301dbb07c3ffea401d99","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.578Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b14add1f63a26f6a5474a163: start","job_status":"start","queue_duration_s":6.597472,"scheduling_latency_s":6.597516}1287gitlab # [ 312.696437] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.203Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1288gitlab # [ 312.700088] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b14add1f63a26f6a5474a163","created_at":"2026-11-10T11:04:06.535Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5275c55bed301dbb07c3ffea401d99","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.578Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b14add1f63a26f6a5474a163: done: 0.044144 sec","job_status":"done","queue_duration_s":6.597472,"scheduling_latency_s":6.597516,"redis_calls":10,"redis_duration_s":0.00374,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000857,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00208,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000803,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00083082,"duration_s":0.044144,"completed_at":"2026-11-10T11:04:13.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1289gitlab # [ 312.744622] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.223Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"afef20a3ac3d3f4f3eabd5fa","created_at":"2026-11-10T11:04:06.611Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19052bf8ea89e45670a584844f7ff704","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.649Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-afef20a3ac3d3f4f3eabd5fa: start","job_status":"start","queue_duration_s":6.573807,"scheduling_latency_s":6.573853}1290gitlab # [ 312.765350] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.316Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"afef20a3ac3d3f4f3eabd5fa","created_at":"2026-11-10T11:04:06.611Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19052bf8ea89e45670a584844f7ff704","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.649Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-afef20a3ac3d3f4f3eabd5fa: done: 0.093278 sec","job_status":"done","queue_duration_s":6.573807,"scheduling_latency_s":6.573853,"redis_calls":14,"redis_duration_s":0.004997,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00133,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00120634,"exclusive_lock_hold_duration_s":0.0425139,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.093278,"completed_at":"2026-11-10T11:04:13.316Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020462,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 312.812964] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa8ed42a7cadb49a9de1064c","created_at":"2026-11-10T11:04:06.682Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b1784b88e834807bc8df1f7fc69e3cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.730Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa8ed42a7cadb49a9de1064c: start","job_status":"start","queue_duration_s":6.589404,"scheduling_latency_s":6.589456}1292gitlab # [ 312.828517] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.349Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1293gitlab # [ 312.831870] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.366Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa8ed42a7cadb49a9de1064c","created_at":"2026-11-10T11:04:06.682Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b1784b88e834807bc8df1f7fc69e3cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.730Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa8ed42a7cadb49a9de1064c: done: 0.045982 sec","job_status":"done","queue_duration_s":6.589404,"scheduling_latency_s":6.589456,"redis_calls":10,"redis_duration_s":0.004048,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000889,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00092366,"duration_s":0.045982,"completed_at":"2026-11-10T11:04:13.365Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00122,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 312.879071] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.369Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"1d47f07147184ffea469a6ae","created_at":"2026-11-10T11:04:06.796Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.848Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-1d47f07147184ffea469a6ae: start","job_status":"start","queue_duration_s":6.521055,"scheduling_latency_s":6.521098}1295gitlab # [ 313.013286] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"1d47f07147184ffea469a6ae","created_at":"2026-11-10T11:04:06.796Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.848Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-1d47f07147184ffea469a6ae: done: 0.195569 sec","job_status":"done","queue_duration_s":6.521055,"scheduling_latency_s":6.521098,"redis_calls":71,"redis_duration_s":0.02138,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.00422,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000856,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.015427,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.173353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032944,"duration_s":0.195569,"completed_at":"2026-11-10T11:04:13.564Z","load_balancing_strategy":"primary","db_duration_s":0.004223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 313.059422] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.567Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"39690c1590edfcca2bd893c3","created_at":"2026-11-10T11:04:06.889Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e252ff3022893dcc25ae82589276d779","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.930Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-39690c1590edfcca2bd893c3: start","job_status":"start","queue_duration_s":6.636438,"scheduling_latency_s":6.636471}1297gitlab # [ 313.078876] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.630Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"39690c1590edfcca2bd893c3","created_at":"2026-11-10T11:04:06.889Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e252ff3022893dcc25ae82589276d779","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:06.930Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-39690c1590edfcca2bd893c3: done: 0.063183 sec","job_status":"done","queue_duration_s":6.636438,"scheduling_latency_s":6.636471,"redis_calls":16,"redis_duration_s":0.005553,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003493,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049464,"duration_s":0.063183,"completed_at":"2026-11-10T11:04:13.630Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 313.124408] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:13.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"688979fca67fe21a100123e3","created_at":"2026-11-10T11:04:06.967Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5952a2db295e4936151176bd76f4b774","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:07.005Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-688979fca67fe21a100123e3: start","job_status":"start","queue_duration_s":6.627794,"scheduling_latency_s":6.627836}1299gitlab # [ 313.446312] postgres[861]: [861] LOG: checkpoint starting: time1300gitlab # [ 313.498670] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"688979fca67fe21a100123e3","created_at":"2026-11-10T11:04:06.967Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5952a2db295e4936151176bd76f4b774","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:04:07.005Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-688979fca67fe21a100123e3: done: 0.416388 sec","job_status":"done","queue_duration_s":6.627794,"scheduling_latency_s":6.627836,"redis_calls":10,"redis_duration_s":0.003245,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000637,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001841,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000767,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056704,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.416388,"completed_at":"2026-11-10T11:04:14.049Z","load_balancing_strategy":"primary","db_duration_s":0.359095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 313.562731] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"debf67e153ad889962bf400f","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-debf67e153ad889962bf400f: start","job_status":"start","queue_duration_s":0.497964,"scheduling_latency_s":0.498007}1302gitlab # [ 313.576782] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"debf67e153ad889962bf400f","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-debf67e153ad889962bf400f: done: 0.059946 sec","job_status":"done","queue_duration_s":0.497964,"scheduling_latency_s":0.498007,"redis_calls":17,"redis_duration_s":0.007274,"redis_read_bytes":142,"redis_write_bytes":1608,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001086,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":371,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003179,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068334,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.059946,"completed_at":"2026-11-10T11:04:14.111Z","load_balancing_strategy":"primary","db_duration_s":0.001199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 313.619885] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8250a24e8a8a0f7843abab55","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8250a24e8a8a0f7843abab55: start","job_status":"start","queue_duration_s":0.561156,"scheduling_latency_s":0.561199}1304gitlab # [ 313.633134] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8250a24e8a8a0f7843abab55","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8250a24e8a8a0f7843abab55: done: 0.041209 sec","job_status":"done","queue_duration_s":0.561156,"scheduling_latency_s":0.561199,"redis_calls":13,"redis_duration_s":0.005721,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00148,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041209,"completed_at":"2026-11-10T11:04:14.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 313.675101] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66fd1c332ddff48572cff180","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66fd1c332ddff48572cff180: start","job_status":"start","queue_duration_s":0.606049,"scheduling_latency_s":0.606088}1306gitlab # [ 313.688245] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66fd1c332ddff48572cff180","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66fd1c332ddff48572cff180: done: 0.030493 sec","job_status":"done","queue_duration_s":0.606049,"scheduling_latency_s":0.606088,"redis_calls":13,"redis_duration_s":0.003171,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030493,"completed_at":"2026-11-10T11:04:14.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 313.729549] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57238013c9cd2bb8a204c4c6","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57238013c9cd2bb8a204c4c6: start","job_status":"start","queue_duration_s":0.63867,"scheduling_latency_s":0.638705}1308gitlab # [ 313.742674] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57238013c9cd2bb8a204c4c6","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57238013c9cd2bb8a204c4c6: done: 0.032003 sec","job_status":"done","queue_duration_s":0.63867,"scheduling_latency_s":0.638705,"redis_calls":13,"redis_duration_s":0.003727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032003,"completed_at":"2026-11-10T11:04:14.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 313.783986] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0d3dec66cf997ef2d6fa6f1","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0d3dec66cf997ef2d6fa6f1: start","job_status":"start","queue_duration_s":0.672722,"scheduling_latency_s":0.672755}1310gitlab # [ 313.796985] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0d3dec66cf997ef2d6fa6f1","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0d3dec66cf997ef2d6fa6f1: done: 0.032364 sec","job_status":"done","queue_duration_s":0.672722,"scheduling_latency_s":0.672755,"redis_calls":13,"redis_duration_s":0.0034399999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032364,"completed_at":"2026-11-10T11:04:14.258Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 313.838040] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b6a7eea3c8aeac659a3dccc","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b6a7eea3c8aeac659a3dccc: start","job_status":"start","queue_duration_s":0.707422,"scheduling_latency_s":0.70746}1312gitlab # [ 313.851175] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b6a7eea3c8aeac659a3dccc","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b6a7eea3c8aeac659a3dccc: done: 0.031035 sec","job_status":"done","queue_duration_s":0.707422,"scheduling_latency_s":0.70746,"redis_calls":13,"redis_duration_s":0.002987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031035,"completed_at":"2026-11-10T11:04:14.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 313.893156] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46cbfd8248facbd38e1ebd4c","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46cbfd8248facbd38e1ebd4c: start","job_status":"start","queue_duration_s":0.742454,"scheduling_latency_s":0.742493}1314gitlab # [ 313.906161] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46cbfd8248facbd38e1ebd4c","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46cbfd8248facbd38e1ebd4c: done: 0.033647 sec","job_status":"done","queue_duration_s":0.742454,"scheduling_latency_s":0.742493,"redis_calls":13,"redis_duration_s":0.003146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001227,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033647,"completed_at":"2026-11-10T11:04:14.329Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 313.947376] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4233fcdca3cd769cc13d674","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4233fcdca3cd769cc13d674: start","job_status":"start","queue_duration_s":0.778416,"scheduling_latency_s":0.778451}1316gitlab # [ 313.960456] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4233fcdca3cd769cc13d674","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4233fcdca3cd769cc13d674: done: 0.025219 sec","job_status":"done","queue_duration_s":0.778416,"scheduling_latency_s":0.778451,"redis_calls":13,"redis_duration_s":0.002407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025219,"completed_at":"2026-11-10T11:04:14.357Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 314.001336] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d4e9aa0e4e63582fe52914","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d4e9aa0e4e63582fe52914: start","job_status":"start","queue_duration_s":0.805182,"scheduling_latency_s":0.805211}1318gitlab # [ 314.014366] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d4e9aa0e4e63582fe52914","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d4e9aa0e4e63582fe52914: done: 0.021008 sec","job_status":"done","queue_duration_s":0.805182,"scheduling_latency_s":0.805211,"redis_calls":13,"redis_duration_s":0.001979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021008,"completed_at":"2026-11-10T11:04:14.380Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 314.055268] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed57929985c0844d1a9c2796","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed57929985c0844d1a9c2796: start","job_status":"start","queue_duration_s":0.8278,"scheduling_latency_s":0.827829}1320gitlab # [ 314.069367] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed57929985c0844d1a9c2796","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed57929985c0844d1a9c2796: done: 0.024152 sec","job_status":"done","queue_duration_s":0.8278,"scheduling_latency_s":0.827829,"redis_calls":13,"redis_duration_s":0.002567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001499,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024152,"completed_at":"2026-11-10T11:04:14.405Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 314.110242] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50679704cc2d07ff0b53b7b5","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50679704cc2d07ff0b53b7b5: start","job_status":"start","queue_duration_s":0.85571,"scheduling_latency_s":0.855748}1322gitlab # [ 314.123315] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50679704cc2d07ff0b53b7b5","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50679704cc2d07ff0b53b7b5: done: 0.03314 sec","job_status":"done","queue_duration_s":0.85571,"scheduling_latency_s":0.855748,"redis_calls":13,"redis_duration_s":0.003285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001345,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03314,"completed_at":"2026-11-10T11:04:14.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 314.164525] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a9bdac99fa05f1033af21a3","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a9bdac99fa05f1033af21a3: start","job_status":"start","queue_duration_s":0.891791,"scheduling_latency_s":0.891836}1324gitlab # [ 314.177353] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a9bdac99fa05f1033af21a3","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a9bdac99fa05f1033af21a3: done: 0.034609 sec","job_status":"done","queue_duration_s":0.891791,"scheduling_latency_s":0.891836,"redis_calls":13,"redis_duration_s":0.003214,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034609,"completed_at":"2026-11-10T11:04:14.480Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 314.218272] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0ac27e7bea457bf19e4002c","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0ac27e7bea457bf19e4002c: start","job_status":"start","queue_duration_s":0.928537,"scheduling_latency_s":0.928574}1326gitlab # [ 314.231166] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0ac27e7bea457bf19e4002c","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0ac27e7bea457bf19e4002c: done: 0.030415 sec","job_status":"done","queue_duration_s":0.928537,"scheduling_latency_s":0.928574,"redis_calls":13,"redis_duration_s":0.002931,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030415,"completed_at":"2026-11-10T11:04:14.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 314.273424] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c12885684ef30bb6f14f185a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c12885684ef30bb6f14f185a: start","job_status":"start","queue_duration_s":0.962762,"scheduling_latency_s":0.962802}1328gitlab # [ 314.286458] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c12885684ef30bb6f14f185a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c12885684ef30bb6f14f185a: done: 0.028223 sec","job_status":"done","queue_duration_s":0.962762,"scheduling_latency_s":0.962802,"redis_calls":13,"redis_duration_s":0.0029490000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028223,"completed_at":"2026-11-10T11:04:14.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 314.328558] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1961437520f980f3cf101f6","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1961437520f980f3cf101f6: start","job_status":"start","queue_duration_s":0.992542,"scheduling_latency_s":0.992569}1330gitlab # [ 314.341906] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1961437520f980f3cf101f6","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1961437520f980f3cf101f6: done: 0.020576 sec","job_status":"done","queue_duration_s":0.992542,"scheduling_latency_s":0.992569,"redis_calls":13,"redis_duration_s":0.002166,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001422,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020576,"completed_at":"2026-11-10T11:04:14.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 314.383245] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32d220019dbb5b0d725c0320","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32d220019dbb5b0d725c0320: start","job_status":"start","queue_duration_s":1.014716,"scheduling_latency_s":1.014744}1332gitlab # [ 314.396354] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32d220019dbb5b0d725c0320","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32d220019dbb5b0d725c0320: done: 0.020522 sec","job_status":"done","queue_duration_s":1.014716,"scheduling_latency_s":1.014744,"redis_calls":13,"redis_duration_s":0.002415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001608,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020522,"completed_at":"2026-11-10T11:04:14.589Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 314.437133] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d388dea63b376c6f6b1053a6","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d388dea63b376c6f6b1053a6: start","job_status":"start","queue_duration_s":1.036863,"scheduling_latency_s":1.036889}1334gitlab # [ 314.450125] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d388dea63b376c6f6b1053a6","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d388dea63b376c6f6b1053a6: done: 0.021657 sec","job_status":"done","queue_duration_s":1.036863,"scheduling_latency_s":1.036889,"redis_calls":13,"redis_duration_s":0.00262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001669,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021657,"completed_at":"2026-11-10T11:04:14.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 314.492200] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb0a4418d6b8369fc7242572","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb0a4418d6b8369fc7242572: start","job_status":"start","queue_duration_s":1.060185,"scheduling_latency_s":1.060211}1336gitlab # [ 314.505438] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb0a4418d6b8369fc7242572","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb0a4418d6b8369fc7242572: done: 0.020073 sec","job_status":"done","queue_duration_s":1.060185,"scheduling_latency_s":1.060211,"redis_calls":13,"redis_duration_s":0.002632,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020073,"completed_at":"2026-11-10T11:04:14.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 314.546364] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"039972ba0ba9ad2e5762cd93","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-039972ba0ba9ad2e5762cd93: start","job_status":"start","queue_duration_s":1.081935,"scheduling_latency_s":1.081961}1338gitlab # [ 314.559538] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"039972ba0ba9ad2e5762cd93","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-039972ba0ba9ad2e5762cd93: done: 0.020618 sec","job_status":"done","queue_duration_s":1.081935,"scheduling_latency_s":1.081961,"redis_calls":13,"redis_duration_s":0.002382,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001561,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020618,"completed_at":"2026-11-10T11:04:14.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 314.601032] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec43e4c1dc7451ae8043a91","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec43e4c1dc7451ae8043a91: start","job_status":"start","queue_duration_s":1.10416,"scheduling_latency_s":1.104187}1340gitlab # [ 314.614033] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec43e4c1dc7451ae8043a91","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec43e4c1dc7451ae8043a91: done: 0.020158 sec","job_status":"done","queue_duration_s":1.10416,"scheduling_latency_s":1.104187,"redis_calls":13,"redis_duration_s":0.002509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001704,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020158,"completed_at":"2026-11-10T11:04:14.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 314.655230] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43589aef7bbd1f471cbda09c","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43589aef7bbd1f471cbda09c: start","job_status":"start","queue_duration_s":1.126556,"scheduling_latency_s":1.126582}1342gitlab # [ 314.668334] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43589aef7bbd1f471cbda09c","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43589aef7bbd1f471cbda09c: done: 0.02023 sec","job_status":"done","queue_duration_s":1.126556,"scheduling_latency_s":1.126582,"redis_calls":13,"redis_duration_s":0.002527,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02023,"completed_at":"2026-11-10T11:04:14.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 314.712549] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15737d401d1f6967685aa7fe","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15737d401d1f6967685aa7fe: start","job_status":"start","queue_duration_s":1.148394,"scheduling_latency_s":1.148419}1344gitlab # [ 314.725467] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15737d401d1f6967685aa7fe","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15737d401d1f6967685aa7fe: done: 0.019585 sec","job_status":"done","queue_duration_s":1.148394,"scheduling_latency_s":1.148419,"redis_calls":13,"redis_duration_s":0.00246,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001608,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019585,"completed_at":"2026-11-10T11:04:14.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 314.766390] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7634dd2ebf931164e717d308","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7634dd2ebf931164e717d308: start","job_status":"start","queue_duration_s":1.169654,"scheduling_latency_s":1.16968}1346gitlab # [ 314.779368] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7634dd2ebf931164e717d308","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7634dd2ebf931164e717d308: done: 0.020008 sec","job_status":"done","queue_duration_s":1.169654,"scheduling_latency_s":1.16968,"redis_calls":13,"redis_duration_s":0.002463,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020008,"completed_at":"2026-11-10T11:04:14.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 314.820679] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d956dd0bafc5c83f75b996fb","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d956dd0bafc5c83f75b996fb: start","job_status":"start","queue_duration_s":1.191612,"scheduling_latency_s":1.191637}1348gitlab # [ 314.833637] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d956dd0bafc5c83f75b996fb","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d956dd0bafc5c83f75b996fb: done: 0.020469 sec","job_status":"done","queue_duration_s":1.191612,"scheduling_latency_s":1.191637,"redis_calls":13,"redis_duration_s":0.002402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001502,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020469,"completed_at":"2026-11-10T11:04:14.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 314.874734] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a9f0047d8b03bfde7d30e69","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a9f0047d8b03bfde7d30e69: start","job_status":"start","queue_duration_s":1.21372,"scheduling_latency_s":1.213747}1350gitlab # [ 314.888896] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a9f0047d8b03bfde7d30e69","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a9f0047d8b03bfde7d30e69: done: 0.01961 sec","job_status":"done","queue_duration_s":1.21372,"scheduling_latency_s":1.213747,"redis_calls":13,"redis_duration_s":0.0023769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001589,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01961,"completed_at":"2026-11-10T11:04:14.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 314.930257] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66cbfa9e80e9c5c448934de0","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66cbfa9e80e9c5c448934de0: start","job_status":"start","queue_duration_s":1.235057,"scheduling_latency_s":1.235086}1352gitlab # [ 314.943288] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66cbfa9e80e9c5c448934de0","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66cbfa9e80e9c5c448934de0: done: 0.020377 sec","job_status":"done","queue_duration_s":1.235057,"scheduling_latency_s":1.235086,"redis_calls":13,"redis_duration_s":0.002218,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001398,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020377,"completed_at":"2026-11-10T11:04:14.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 314.984785] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0a9c1ec8f3a4cf3b082d00e","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0a9c1ec8f3a4cf3b082d00e: start","job_status":"start","queue_duration_s":1.257046,"scheduling_latency_s":1.257073}1354gitlab # [ 314.997746] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0a9c1ec8f3a4cf3b082d00e","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0a9c1ec8f3a4cf3b082d00e: done: 0.020351 sec","job_status":"done","queue_duration_s":1.257046,"scheduling_latency_s":1.257073,"redis_calls":13,"redis_duration_s":0.002548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001627,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020351,"completed_at":"2026-11-10T11:04:14.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 315.038897] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2df77c17f8d9ae7313744a2","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2df77c17f8d9ae7313744a2: start","job_status":"start","queue_duration_s":1.27908,"scheduling_latency_s":1.279107}1356gitlab # [ 315.052155] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2df77c17f8d9ae7313744a2","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2df77c17f8d9ae7313744a2: done: 0.02103 sec","job_status":"done","queue_duration_s":1.27908,"scheduling_latency_s":1.279107,"redis_calls":13,"redis_duration_s":0.002515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001567,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02103,"completed_at":"2026-11-10T11:04:14.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 315.094378] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c41e6efb36cc5d2b3c498807","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c41e6efb36cc5d2b3c498807: start","job_status":"start","queue_duration_s":1.301912,"scheduling_latency_s":1.301942}1358gitlab # [ 315.107394] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c41e6efb36cc5d2b3c498807","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c41e6efb36cc5d2b3c498807: done: 0.020539 sec","job_status":"done","queue_duration_s":1.301912,"scheduling_latency_s":1.301942,"redis_calls":13,"redis_duration_s":0.0030350000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020539,"completed_at":"2026-11-10T11:04:14.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 315.148598] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcfd1863b0eb68965c572c84","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcfd1863b0eb68965c572c84: start","job_status":"start","queue_duration_s":1.324308,"scheduling_latency_s":1.324337}1360gitlab # [ 315.161538] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcfd1863b0eb68965c572c84","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcfd1863b0eb68965c572c84: done: 0.020747 sec","job_status":"done","queue_duration_s":1.324308,"scheduling_latency_s":1.324337,"redis_calls":13,"redis_duration_s":0.002541,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001651,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020747,"completed_at":"2026-11-10T11:04:14.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 315.202662] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de8c828a2366cfa923ccdffc","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de8c828a2366cfa923ccdffc: start","job_status":"start","queue_duration_s":1.346909,"scheduling_latency_s":1.346935}1362gitlab # [ 315.215903] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de8c828a2366cfa923ccdffc","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de8c828a2366cfa923ccdffc: done: 0.018374 sec","job_status":"done","queue_duration_s":1.346909,"scheduling_latency_s":1.346935,"redis_calls":13,"redis_duration_s":0.002138,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001397,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018374,"completed_at":"2026-11-10T11:04:14.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 315.257405] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648a1ee0ee7913ce82f245ac","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648a1ee0ee7913ce82f245ac: start","job_status":"start","queue_duration_s":1.367371,"scheduling_latency_s":1.367408}1364gitlab # [ 315.270412] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"648a1ee0ee7913ce82f245ac","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-648a1ee0ee7913ce82f245ac: done: 0.033335 sec","job_status":"done","queue_duration_s":1.367371,"scheduling_latency_s":1.367408,"redis_calls":13,"redis_duration_s":0.0035440000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002363,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033335,"completed_at":"2026-11-10T11:04:14.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 315.313050] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93560cbcb9ff06ac2b86dcaa","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93560cbcb9ff06ac2b86dcaa: start","job_status":"start","queue_duration_s":1.403328,"scheduling_latency_s":1.403372}1366gitlab # [ 315.325744] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93560cbcb9ff06ac2b86dcaa","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93560cbcb9ff06ac2b86dcaa: done: 0.034544 sec","job_status":"done","queue_duration_s":1.403328,"scheduling_latency_s":1.403372,"redis_calls":13,"redis_duration_s":0.003576,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00134,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034544,"completed_at":"2026-11-10T11:04:14.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 315.366711] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:14.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12602d3b0c1b02549eb9775b","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12602d3b0c1b02549eb9775b: start","job_status":"start","queue_duration_s":1.440223,"scheduling_latency_s":1.440262}1368gitlab # [ 315.379782] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12602d3b0c1b02549eb9775b","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12602d3b0c1b02549eb9775b: done: 0.032107 sec","job_status":"done","queue_duration_s":1.440223,"scheduling_latency_s":1.440262,"redis_calls":13,"redis_duration_s":0.003209,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032107,"completed_at":"2026-11-10T11:04:15.026Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 315.421102] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"731053d8cf971363cdfc263a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-731053d8cf971363cdfc263a: start","job_status":"start","queue_duration_s":1.474803,"scheduling_latency_s":1.474845}1370gitlab # [ 315.434061] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"731053d8cf971363cdfc263a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-731053d8cf971363cdfc263a: done: 0.028538 sec","job_status":"done","queue_duration_s":1.474803,"scheduling_latency_s":1.474845,"redis_calls":13,"redis_duration_s":0.002524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028538,"completed_at":"2026-11-10T11:04:15.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 315.474910] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43a54c5d8f519700298f2334","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43a54c5d8f519700298f2334: start","job_status":"start","queue_duration_s":1.505636,"scheduling_latency_s":1.505675}1372gitlab # [ 315.487925] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43a54c5d8f519700298f2334","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43a54c5d8f519700298f2334: done: 0.029085 sec","job_status":"done","queue_duration_s":1.505636,"scheduling_latency_s":1.505675,"redis_calls":13,"redis_duration_s":0.003051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001199,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029085,"completed_at":"2026-11-10T11:04:15.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 315.530514] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58aae79aa06d75f19cb2a6e7","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58aae79aa06d75f19cb2a6e7: start","job_status":"start","queue_duration_s":1.536995,"scheduling_latency_s":1.537034}1374gitlab # [ 315.543622] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58aae79aa06d75f19cb2a6e7","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58aae79aa06d75f19cb2a6e7: done: 0.028101 sec","job_status":"done","queue_duration_s":1.536995,"scheduling_latency_s":1.537034,"redis_calls":13,"redis_duration_s":0.002835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001023,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001812,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028101,"completed_at":"2026-11-10T11:04:15.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 315.584943] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6ee2f9ca889c88d8664301a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6ee2f9ca889c88d8664301a: start","job_status":"start","queue_duration_s":1.567075,"scheduling_latency_s":1.567101}1376gitlab # [ 315.597684] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6ee2f9ca889c88d8664301a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6ee2f9ca889c88d8664301a: done: 0.02144 sec","job_status":"done","queue_duration_s":1.567075,"scheduling_latency_s":1.567101,"redis_calls":13,"redis_duration_s":0.00211,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001289,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02144,"completed_at":"2026-11-10T11:04:15.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 315.638651] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e68accb17c9e1d108ac64393","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e68accb17c9e1d108ac64393: start","job_status":"start","queue_duration_s":1.590757,"scheduling_latency_s":1.590824}1378gitlab # [ 315.652166] gitaly[923]: time="2026-11-10T11:04:16.145Z" level=info msg="burden monitor stats" pid=9231379gitlab # [ 315.653488] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e68accb17c9e1d108ac64393","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e68accb17c9e1d108ac64393: done: 0.024688 sec","job_status":"done","queue_duration_s":1.590757,"scheduling_latency_s":1.590824,"redis_calls":13,"redis_duration_s":0.002918,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024688,"completed_at":"2026-11-10T11:04:15.169Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 315.694827] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c40e34f57bbe63ad1a73aab","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c40e34f57bbe63ad1a73aab: start","job_status":"start","queue_duration_s":1.617625,"scheduling_latency_s":1.617655}1381gitlab # [ 315.708880] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c40e34f57bbe63ad1a73aab","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c40e34f57bbe63ad1a73aab: done: 0.024537 sec","job_status":"done","queue_duration_s":1.617625,"scheduling_latency_s":1.617655,"redis_calls":13,"redis_duration_s":0.00262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024537,"completed_at":"2026-11-10T11:04:15.196Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 315.749607] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4093eebca566a7dbe1ebb8d","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4093eebca566a7dbe1ebb8d: start","job_status":"start","queue_duration_s":1.645503,"scheduling_latency_s":1.645543}1383gitlab # [ 315.762573] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4093eebca566a7dbe1ebb8d","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4093eebca566a7dbe1ebb8d: done: 0.035944 sec","job_status":"done","queue_duration_s":1.645503,"scheduling_latency_s":1.645543,"redis_calls":13,"redis_duration_s":0.003948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002408,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035944,"completed_at":"2026-11-10T11:04:15.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 315.803594] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcfe8139a48097a0aeedc73a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcfe8139a48097a0aeedc73a: start","job_status":"start","queue_duration_s":1.684674,"scheduling_latency_s":1.684715}1385gitlab # [ 315.816721] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcfe8139a48097a0aeedc73a","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcfe8139a48097a0aeedc73a: done: 0.032503 sec","job_status":"done","queue_duration_s":1.684674,"scheduling_latency_s":1.684715,"redis_calls":13,"redis_duration_s":0.00375,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032503,"completed_at":"2026-11-10T11:04:15.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 315.857359] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d02565f36b9778fb498ec3c2","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d02565f36b9778fb498ec3c2: start","job_status":"start","queue_duration_s":1.718964,"scheduling_latency_s":1.718998}1387gitlab # [ 315.869878] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.295Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d02565f36b9778fb498ec3c2","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d02565f36b9778fb498ec3c2: done: 0.022878 sec","job_status":"done","queue_duration_s":1.718964,"scheduling_latency_s":1.718998,"redis_calls":13,"redis_duration_s":0.002257,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022878,"completed_at":"2026-11-10T11:04:15.295Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 315.911716] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f7eba8b43b4f99f1a7ad60","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f7eba8b43b4f99f1a7ad60: start","job_status":"start","queue_duration_s":1.743558,"scheduling_latency_s":1.743585}1389gitlab # [ 315.924708] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f7eba8b43b4f99f1a7ad60","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f7eba8b43b4f99f1a7ad60: done: 0.021304 sec","job_status":"done","queue_duration_s":1.743558,"scheduling_latency_s":1.743585,"redis_calls":13,"redis_duration_s":0.002084,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021304,"completed_at":"2026-11-10T11:04:15.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 315.965458] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ceeb077ffaed9ad9dfa510b8","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ceeb077ffaed9ad9dfa510b8: start","job_status":"start","queue_duration_s":1.766738,"scheduling_latency_s":1.766766}1391gitlab # [ 315.978483] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ceeb077ffaed9ad9dfa510b8","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ceeb077ffaed9ad9dfa510b8: done: 0.021536 sec","job_status":"done","queue_duration_s":1.766738,"scheduling_latency_s":1.766766,"redis_calls":13,"redis_duration_s":0.002265,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001349,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021536,"completed_at":"2026-11-10T11:04:15.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 316.019839] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0051a9aa087807f556b7567","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0051a9aa087807f556b7567: start","job_status":"start","queue_duration_s":1.789973,"scheduling_latency_s":1.790001}1393gitlab # [ 316.033049] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0051a9aa087807f556b7567","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0051a9aa087807f556b7567: done: 0.021872 sec","job_status":"done","queue_duration_s":1.789973,"scheduling_latency_s":1.790001,"redis_calls":13,"redis_duration_s":0.002353,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001498,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021872,"completed_at":"2026-11-10T11:04:15.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 316.073983] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1513244ba03f550b1a56b516","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1513244ba03f550b1a56b516: start","job_status":"start","queue_duration_s":1.813423,"scheduling_latency_s":1.813449}1395gitlab # [ 316.087029] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1513244ba03f550b1a56b516","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1513244ba03f550b1a56b516: done: 0.020725 sec","job_status":"done","queue_duration_s":1.813423,"scheduling_latency_s":1.813449,"redis_calls":13,"redis_duration_s":0.00224,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001412,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020725,"completed_at":"2026-11-10T11:04:15.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 316.129124] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d566db4d201e72b353865618","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d566db4d201e72b353865618: start","job_status":"start","queue_duration_s":1.835811,"scheduling_latency_s":1.835839}1397gitlab # [ 316.142867] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d566db4d201e72b353865618","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d566db4d201e72b353865618: done: 0.021973 sec","job_status":"done","queue_duration_s":1.835811,"scheduling_latency_s":1.835839,"redis_calls":13,"redis_duration_s":0.0027359999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021973,"completed_at":"2026-11-10T11:04:15.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 316.184304] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442adf8f2b4113352235c87d","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442adf8f2b4113352235c87d: start","job_status":"start","queue_duration_s":1.859737,"scheduling_latency_s":1.859765}1399gitlab # [ 316.197536] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442adf8f2b4113352235c87d","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442adf8f2b4113352235c87d: done: 0.023221 sec","job_status":"done","queue_duration_s":1.859737,"scheduling_latency_s":1.859765,"redis_calls":13,"redis_duration_s":0.002856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023221,"completed_at":"2026-11-10T11:04:15.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 316.238695] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6355993ae8924789ff64c9c7","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6355993ae8924789ff64c9c7: start","job_status":"start","queue_duration_s":1.884588,"scheduling_latency_s":1.884617}1401gitlab # [ 316.251729] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:04:15.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6355993ae8924789ff64c9c7","created_at":"2026-11-10T11:04:13.423Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-06526889cfd845c0","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"cd34afb472e3540b1d66acc4938d2642","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:04:13.554Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6355993ae8924789ff64c9c7: done: 0.024179 sec","job_status":"done","queue_duration_s":1.884588,"scheduling_latency_s":1.884617,"redis_calls":13,"redis_duration_s":0.0024939999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001499,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024179,"completed_at":"2026-11-10T11:04:15.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 363.822288] setup-gitlab-start[3573]: W, [2026-11-10T11:05:04.373537 #3573] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1403gitlab # [ 365.074479] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:05.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"583e6c64c771773202473e64","created_at":"2026-11-10T11:05:05.572Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.622Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-583e6c64c771773202473e64: start","job_status":"start","queue_duration_s":0.003183,"scheduling_latency_s":0.00323}1404gitlab # [ 365.252562] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:05.803Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"583e6c64c771773202473e64","created_at":"2026-11-10T11:05:05.572Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.622Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-583e6c64c771773202473e64: done: 0.178442 sec","job_status":"done","queue_duration_s":0.003183,"scheduling_latency_s":0.00323,"redis_calls":24,"redis_duration_s":0.032645,"redis_read_bytes":1171,"redis_write_bytes":4682,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.010376,"redis_feature_flag_read_bytes":1156,"redis_feature_flag_write_bytes":652,"redis_queues_calls":2,"redis_queues_duration_s":0.000562,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.017817,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.178442,"completed_at":"2026-11-10T11:05:05.803Z","load_balancing_strategy":"primary","db_duration_s":0.003222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 365.316454] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:05.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"849f8b67c305c8348bab1fa6","created_at":"2026-11-10T11:05:05.635Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d5cd81ae78d5aa321c1fb46cbd1c6e0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.666Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-849f8b67c305c8348bab1fa6: start","job_status":"start","queue_duration_s":0.144875,"scheduling_latency_s":0.144921}1406gitlab # [ 366.282592] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"849f8b67c305c8348bab1fa6","created_at":"2026-11-10T11:05:05.635Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d5cd81ae78d5aa321c1fb46cbd1c6e0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.666Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-849f8b67c305c8348bab1fa6: done: 1.022171 sec","job_status":"done","queue_duration_s":0.144875,"scheduling_latency_s":0.144921,"redis_calls":10,"redis_duration_s":0.004698,"redis_read_bytes":135,"redis_write_bytes":1071,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":84,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.182,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.182,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.526381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00234386,"exclusive_lock_hold_duration_s":0.97690078,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":1.022171,"completed_at":"2026-11-10T11:05:06.833Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.525818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 366.283672] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ac7a1e039ec46e83a0352189","created_at":"2026-11-10T11:05:05.684Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b0e6d8515f93cf588ac33d6cd515907c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.703Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ac7a1e039ec46e83a0352189: start","job_status":"start","queue_duration_s":1.131876,"scheduling_latency_s":1.131911}1408gitlab # [ 366.371850] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.921Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1409gitlab # [ 366.383391] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.934Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ac7a1e039ec46e83a0352189","created_at":"2026-11-10T11:05:05.684Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b0e6d8515f93cf588ac33d6cd515907c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.703Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ac7a1e039ec46e83a0352189: done: 0.099273 sec","job_status":"done","queue_duration_s":1.131876,"scheduling_latency_s":1.131911,"redis_calls":7,"redis_duration_s":0.0031929999999999997,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099273,"completed_at":"2026-11-10T11:05:06.934Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 366.429033] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.937Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8572983445ce412894854560","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8572983445ce412894854560: start","job_status":"start","queue_duration_s":1.211366,"scheduling_latency_s":1.211408}1411gitlab # [ 366.442300] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.961Z","message":"No existing merge request to be cleaned up."}1412gitlab # [ 366.444083] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.980Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8572983445ce412894854560","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8572983445ce412894854560: done: 0.042927 sec","job_status":"done","queue_duration_s":1.211366,"scheduling_latency_s":1.211408,"redis_calls":10,"redis_duration_s":0.004401,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001291,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00311,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042927,"completed_at":"2026-11-10T11:05:06.980Z","load_balancing_strategy":"primary","db_duration_s":0.004777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 366.484611] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:06.982Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1fc5659ad8bc97f6d34de6b5","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1fc5659ad8bc97f6d34de6b5: start","job_status":"start","queue_duration_s":1.256621,"scheduling_latency_s":1.256654}1414gitlab # [ 366.497563] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.004Z","message":"No existing merge request to be cleaned up."}1415gitlab # [ 366.499273] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.020Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1fc5659ad8bc97f6d34de6b5","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1fc5659ad8bc97f6d34de6b5: done: 0.038179 sec","job_status":"done","queue_duration_s":1.256621,"scheduling_latency_s":1.256654,"redis_calls":10,"redis_duration_s":0.0028970000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001198,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001699,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038179,"completed_at":"2026-11-10T11:05:07.020Z","load_balancing_strategy":"primary","db_duration_s":0.003373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 366.540277] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.023Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f5481f9d44023d0c49bf5a2","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f5481f9d44023d0c49bf5a2: start","job_status":"start","queue_duration_s":1.297118,"scheduling_latency_s":1.29715}1417gitlab # [ 366.554313] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.043Z","message":"No existing merge request to be cleaned up."}1418gitlab # [ 366.556072] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.059Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f5481f9d44023d0c49bf5a2","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f5481f9d44023d0c49bf5a2: done: 0.036612 sec","job_status":"done","queue_duration_s":1.297118,"scheduling_latency_s":1.29715,"redis_calls":10,"redis_duration_s":0.0030039999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001263,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001741,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036612,"completed_at":"2026-11-10T11:05:07.059Z","load_balancing_strategy":"primary","db_duration_s":0.003182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 366.596673] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.061Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e3cae46054b0637a03db87b1","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e3cae46054b0637a03db87b1: start","job_status":"start","queue_duration_s":1.335833,"scheduling_latency_s":1.33587}1420gitlab # [ 366.609505] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.085Z","message":"No existing merge request to be cleaned up."}1421gitlab # [ 366.611250] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.101Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e3cae46054b0637a03db87b1","created_at":"2026-11-10T11:05:05.683Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-4047cae2a77040a3","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"be68bf9f55771f6681f91fc369131327","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.726Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e3cae46054b0637a03db87b1: done: 0.039205 sec","job_status":"done","queue_duration_s":1.335833,"scheduling_latency_s":1.33587,"redis_calls":10,"redis_duration_s":0.003022,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001735,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039205,"completed_at":"2026-11-10T11:05:07.100Z","load_balancing_strategy":"primary","db_duration_s":0.003016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 366.652075] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"60b35c9aa6993e369dacdde8","created_at":"2026-11-10T11:05:05.771Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"045a15290623299beaadd343913138a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.787Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-60b35c9aa6993e369dacdde8: start","job_status":"start","queue_duration_s":1.316208,"scheduling_latency_s":1.316246}1423gitlab # [ 366.667117] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.127Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"60b35c9aa6993e369dacdde8","created_at":"2026-11-10T11:05:05.771Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"045a15290623299beaadd343913138a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.787Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-60b35c9aa6993e369dacdde8: done: 0.02406 sec","job_status":"done","queue_duration_s":1.316208,"scheduling_latency_s":1.316246,"redis_calls":8,"redis_duration_s":0.002128,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00024,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000542,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02406,"completed_at":"2026-11-10T11:05:07.127Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 366.711639] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"71fd982e9f35fc295a442195","created_at":"2026-11-10T11:05:05.805Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"593aa03270a232cd8e4058260f551097","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.839Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-71fd982e9f35fc295a442195: start","job_status":"start","queue_duration_s":1.289748,"scheduling_latency_s":1.289782}1425gitlab # [ 366.727065] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"71fd982e9f35fc295a442195","created_at":"2026-11-10T11:05:05.805Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"593aa03270a232cd8e4058260f551097","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.839Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-71fd982e9f35fc295a442195: done: 0.036503 sec","job_status":"done","queue_duration_s":1.289748,"scheduling_latency_s":1.289782,"redis_calls":10,"redis_duration_s":0.0030080000000000003,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001686,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00921678,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.036503,"completed_at":"2026-11-10T11:05:07.165Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003777,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1426gitlab # [ 366.773583] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d304abd81b12aeabeea25ca4","created_at":"2026-11-10T11:05:05.883Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b20c321f3021b06edc964c78b576cb1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.890Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d304abd81b12aeabeea25ca4: start","job_status":"start","queue_duration_s":1.278022,"scheduling_latency_s":1.278056}1427gitlab # [ 366.789057] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d304abd81b12aeabeea25ca4","created_at":"2026-11-10T11:05:05.883Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b20c321f3021b06edc964c78b576cb1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.890Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d304abd81b12aeabeea25ca4: done: 0.064999 sec","job_status":"done","queue_duration_s":1.278022,"scheduling_latency_s":1.278056,"redis_calls":9,"redis_duration_s":0.003058,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00139336,"exclusive_lock_hold_duration_s":0.0238134,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.064999,"completed_at":"2026-11-10T11:05:07.232Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 366.834398] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f2208b667233638f6d99318f","created_at":"2026-11-10T11:05:05.900Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"260300172da6d1ea6e2e2df4fbd3fd7e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.910Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f2208b667233638f6d99318f: start","job_status":"start","queue_duration_s":1.324932,"scheduling_latency_s":1.32497}1429gitlab # [ 366.848912] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f2208b667233638f6d99318f","created_at":"2026-11-10T11:05:05.900Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"260300172da6d1ea6e2e2df4fbd3fd7e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.910Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f2208b667233638f6d99318f: done: 0.030614 sec","job_status":"done","queue_duration_s":1.324932,"scheduling_latency_s":1.32497,"redis_calls":9,"redis_duration_s":0.002744,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008465,"exclusive_lock_hold_duration_s":0.00577394,"duration_s":0.030614,"completed_at":"2026-11-10T11:05:07.266Z","load_balancing_strategy":"primary","db_duration_s":0.001671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 366.890849] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"a58cb0d9423f67ca8e9c3308","created_at":"2026-11-10T11:05:05.938Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7debdc6ce1dad0077c6e122defaa33fc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.961Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-a58cb0d9423f67ca8e9c3308: start","job_status":"start","queue_duration_s":1.306373,"scheduling_latency_s":1.306438}1431gitlab # [ 366.905089] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"a58cb0d9423f67ca8e9c3308","created_at":"2026-11-10T11:05:05.938Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7debdc6ce1dad0077c6e122defaa33fc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:05.961Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-a58cb0d9423f67ca8e9c3308: done: 0.041801 sec","job_status":"done","queue_duration_s":1.306373,"scheduling_latency_s":1.306438,"redis_calls":10,"redis_duration_s":0.002674,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001544,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038464,"duration_s":0.041801,"completed_at":"2026-11-10T11:05:07.309Z","load_balancing_strategy":"primary","db_duration_s":0.005247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 366.948463] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.311Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a36b46e2f49a10e88dc518be","created_at":"2026-11-10T11:05:05.986Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a8c8556030a60660938c0c42c9da2d5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.003Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a36b46e2f49a10e88dc518be: start","job_status":"start","queue_duration_s":1.308172,"scheduling_latency_s":1.308197}1433gitlab # [ 366.964717] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a36b46e2f49a10e88dc518be","created_at":"2026-11-10T11:05:05.986Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a8c8556030a60660938c0c42c9da2d5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.003Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a36b46e2f49a10e88dc518be: done: 0.024145 sec","job_status":"done","queue_duration_s":1.308172,"scheduling_latency_s":1.308197,"redis_calls":9,"redis_duration_s":0.002392,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006746,"exclusive_lock_hold_duration_s":0.00455988,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.024145,"completed_at":"2026-11-10T11:05:07.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001735,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 367.008676] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"01ce15f5a7e260845096e279","created_at":"2026-11-10T11:05:06.032Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b06920b857568e810fed2a00c828413","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:05:06.049Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-01ce15f5a7e260845096e279: start","job_status":"start","queue_duration_s":1.288947,"scheduling_latency_s":1.288979}1435gitlab # [ 367.022947] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.349Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1436gitlab # [ 367.026643] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.357Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"01ce15f5a7e260845096e279","created_at":"2026-11-10T11:05:06.032Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b06920b857568e810fed2a00c828413","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:05:06.049Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-01ce15f5a7e260845096e279: done: 0.019061 sec","job_status":"done","queue_duration_s":1.288947,"scheduling_latency_s":1.288979,"redis_calls":7,"redis_duration_s":0.0018,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019061,"completed_at":"2026-11-10T11:05:07.357Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 367.068486] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.359Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0f5315d193138fd5cd4c84f4","created_at":"2026-11-10T11:05:06.074Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c1896b09421d5bd28cc1246a386e9b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:05:06.090Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0f5315d193138fd5cd4c84f4: start","job_status":"start","queue_duration_s":1.268882,"scheduling_latency_s":1.268907}1438gitlab # [ 367.082561] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.370Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1439gitlab # [ 367.086578] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0f5315d193138fd5cd4c84f4","created_at":"2026-11-10T11:05:06.074Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c1896b09421d5bd28cc1246a386e9b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:05:06.090Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0f5315d193138fd5cd4c84f4: done: 0.019879 sec","job_status":"done","queue_duration_s":1.268882,"scheduling_latency_s":1.268907,"redis_calls":7,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000653,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019879,"completed_at":"2026-11-10T11:05:07.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 367.127918] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"2ee3e8bfaa20513472e6c817","created_at":"2026-11-10T11:05:06.112Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"689f23b18a23aefac6ba1a107e73d2d6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.144Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-2ee3e8bfaa20513472e6c817: start","job_status":"start","queue_duration_s":1.23648,"scheduling_latency_s":1.236507}1441gitlab # [ 367.142872] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"2ee3e8bfaa20513472e6c817","created_at":"2026-11-10T11:05:06.112Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"689f23b18a23aefac6ba1a107e73d2d6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.144Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-2ee3e8bfaa20513472e6c817: done: 0.027819 sec","job_status":"done","queue_duration_s":1.23648,"scheduling_latency_s":1.236507,"redis_calls":11,"redis_duration_s":0.002818,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001366,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000397,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034642,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.027819,"completed_at":"2026-11-10T11:05:07.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 367.191713] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"659084f2833154ec70e290a9","created_at":"2026-11-10T11:05:06.176Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1325982ff5304521962242ba751e84a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.193Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-659084f2833154ec70e290a9: start","job_status":"start","queue_duration_s":1.21766,"scheduling_latency_s":1.217687}1443gitlab # [ 367.207406] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.422Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1444gitlab # [ 367.211146] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"659084f2833154ec70e290a9","created_at":"2026-11-10T11:05:06.176Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1325982ff5304521962242ba751e84a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.193Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-659084f2833154ec70e290a9: done: 0.019423 sec","job_status":"done","queue_duration_s":1.21766,"scheduling_latency_s":1.217687,"redis_calls":7,"redis_duration_s":0.00174,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001261,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019423,"completed_at":"2026-11-10T11:05:07.430Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 367.254136] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.432Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24bb98a2b2d2c0489c99ac3a","created_at":"2026-11-10T11:05:06.231Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc00686aabf2cfd469ad576ef37414f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:05:06.244Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24bb98a2b2d2c0489c99ac3a: start","job_status":"start","queue_duration_s":1.187555,"scheduling_latency_s":1.187579}1446gitlab # [ 367.268112] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24bb98a2b2d2c0489c99ac3a","created_at":"2026-11-10T11:05:06.231Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc00686aabf2cfd469ad576ef37414f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:05:06.244Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24bb98a2b2d2c0489c99ac3a: done: 0.032393 sec","job_status":"done","queue_duration_s":1.187555,"scheduling_latency_s":1.187579,"redis_calls":7,"redis_duration_s":0.001999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032393,"completed_at":"2026-11-10T11:05:07.464Z","load_balancing_strategy":"primary","db_duration_s":0.002361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 367.309576] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"08813276eb606d6b911beabb","created_at":"2026-11-10T11:05:06.296Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6de0c4d9da7d4ea23906b050259e5557","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.308Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-08813276eb606d6b911beabb: start","job_status":"start","queue_duration_s":1.158636,"scheduling_latency_s":1.158668}1448gitlab # [ 367.325040] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"08813276eb606d6b911beabb","created_at":"2026-11-10T11:05:06.296Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6de0c4d9da7d4ea23906b050259e5557","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.308Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-08813276eb606d6b911beabb: done: 0.023574 sec","job_status":"done","queue_duration_s":1.158636,"scheduling_latency_s":1.158668,"redis_calls":7,"redis_duration_s":0.001798,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023574,"completed_at":"2026-11-10T11:05:07.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 367.367383] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c7d5f9d36ca09c55389da025","created_at":"2026-11-10T11:05:06.352Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"673655a7f229b2ed1024a7082039e9cf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.373Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c7d5f9d36ca09c55389da025: start","job_status":"start","queue_duration_s":1.119277,"scheduling_latency_s":1.119307}1450gitlab # [ 367.383899] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c7d5f9d36ca09c55389da025","created_at":"2026-11-10T11:05:06.352Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"673655a7f229b2ed1024a7082039e9cf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.373Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c7d5f9d36ca09c55389da025: done: 0.443193 sec","job_status":"done","queue_duration_s":1.119277,"scheduling_latency_s":1.119307,"redis_calls":562,"redis_duration_s":0.106076,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001593,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.104483,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.388578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.443193,"completed_at":"2026-11-10T11:05:07.935Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 367.426418] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.937Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6736a2b990f81565c1b79b61","created_at":"2026-11-10T11:05:06.427Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3dff9f0be3fd8ccfe8c450fc0b2674a0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.450Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6736a2b990f81565c1b79b61: start","job_status":"start","queue_duration_s":1.486431,"scheduling_latency_s":1.486465}1452gitlab # [ 367.441169] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.960Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6736a2b990f81565c1b79b61","created_at":"2026-11-10T11:05:06.427Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3dff9f0be3fd8ccfe8c450fc0b2674a0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.450Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6736a2b990f81565c1b79b61: done: 0.022965 sec","job_status":"done","queue_duration_s":1.486431,"scheduling_latency_s":1.486465,"redis_calls":7,"redis_duration_s":0.0016680000000000002,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022965,"completed_at":"2026-11-10T11:05:07.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 367.483391] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"dc1719926467dcdc4197a573","created_at":"2026-11-10T11:05:06.504Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d17816be8abd04e0097356a979418cb7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.522Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-dc1719926467dcdc4197a573: start","job_status":"start","queue_duration_s":1.439998,"scheduling_latency_s":1.440033}1454gitlab # [ 367.498729] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.974Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1455gitlab # [ 367.502581] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"dc1719926467dcdc4197a573","created_at":"2026-11-10T11:05:06.504Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d17816be8abd04e0097356a979418cb7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.522Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-dc1719926467dcdc4197a573: done: 0.02106 sec","job_status":"done","queue_duration_s":1.439998,"scheduling_latency_s":1.440033,"redis_calls":8,"redis_duration_s":0.0019030000000000002,"redis_read_bytes":133,"redis_write_bytes":894,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":128,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02106,"completed_at":"2026-11-10T11:05:07.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 367.547014] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:07.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1234f3f68cef0913567fb0d2","created_at":"2026-11-10T11:05:06.550Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbcef1d16981372e02aa1bf1bb5a4fdb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.556Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1234f3f68cef0913567fb0d2: start","job_status":"start","queue_duration_s":1.427918,"scheduling_latency_s":1.427948}1457gitlab # [ 367.562448] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1234f3f68cef0913567fb0d2","created_at":"2026-11-10T11:05:06.550Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbcef1d16981372e02aa1bf1bb5a4fdb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.556Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1234f3f68cef0913567fb0d2: done: 0.052806 sec","job_status":"done","queue_duration_s":1.427918,"scheduling_latency_s":1.427948,"redis_calls":10,"redis_duration_s":0.00302,"redis_read_bytes":135,"redis_write_bytes":1294,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":132,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001549,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00077164,"exclusive_lock_hold_duration_s":0.02874322,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.052806,"completed_at":"2026-11-10T11:05:08.037Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 367.610908] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7f9611b2bf3ec1767d5ff171","created_at":"2026-11-10T11:05:06.577Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf0526e5128f5cb52cbab913febf635a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.590Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7f9611b2bf3ec1767d5ff171: start","job_status":"start","queue_duration_s":1.449656,"scheduling_latency_s":1.449687}1459gitlab # [ 367.626155] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.052Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1460gitlab # [ 367.629603] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7f9611b2bf3ec1767d5ff171","created_at":"2026-11-10T11:05:06.577Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf0526e5128f5cb52cbab913febf635a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.590Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7f9611b2bf3ec1767d5ff171: done: 0.020906 sec","job_status":"done","queue_duration_s":1.449656,"scheduling_latency_s":1.449687,"redis_calls":8,"redis_duration_s":0.001977,"redis_read_bytes":133,"redis_write_bytes":900,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000237,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":129,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020906,"completed_at":"2026-11-10T11:05:08.060Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 367.673510] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c7b90e56d9a68309e6ec35d4","created_at":"2026-11-10T11:05:06.631Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.644Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c7b90e56d9a68309e6ec35d4: start","job_status":"start","queue_duration_s":1.417955,"scheduling_latency_s":1.417987}1462gitlab # [ 367.691776] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c7b90e56d9a68309e6ec35d4","created_at":"2026-11-10T11:05:06.631Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.644Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c7b90e56d9a68309e6ec35d4: done: 0.178785 sec","job_status":"done","queue_duration_s":1.417955,"scheduling_latency_s":1.417987,"redis_calls":61,"redis_duration_s":0.017698,"redis_read_bytes":236,"redis_write_bytes":41183,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":124,"redis_queues_calls":2,"redis_queues_duration_s":0.001008,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.015656,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.156783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.178785,"completed_at":"2026-11-10T11:05:08.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 367.736691] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.245Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"cbdeaeca01a1443a161f2270","created_at":"2026-11-10T11:05:06.669Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"217b73b2f5d2bb4a052ba00e6684af1c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.682Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-cbdeaeca01a1443a161f2270: start","job_status":"start","queue_duration_s":1.562754,"scheduling_latency_s":1.562795}1464gitlab # [ 367.751510] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.283Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"cbdeaeca01a1443a161f2270","created_at":"2026-11-10T11:05:06.669Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"217b73b2f5d2bb4a052ba00e6684af1c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.682Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-cbdeaeca01a1443a161f2270: done: 0.038078 sec","job_status":"done","queue_duration_s":1.562754,"scheduling_latency_s":1.562795,"redis_calls":8,"redis_duration_s":0.002293,"redis_read_bytes":133,"redis_write_bytes":827,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.0003,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":116,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001519,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038078,"completed_at":"2026-11-10T11:05:08.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 367.797547] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"cd1f8849427c44f477972fc1","created_at":"2026-11-10T11:05:06.706Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2a469d13feda8948a1ab863266583d5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.719Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-cd1f8849427c44f477972fc1: start","job_status":"start","queue_duration_s":1.567054,"scheduling_latency_s":1.567091}1466gitlab # [ 367.812308] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"cd1f8849427c44f477972fc1","created_at":"2026-11-10T11:05:06.706Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2a469d13feda8948a1ab863266583d5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:05:06.719Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-cd1f8849427c44f477972fc1: done: 0.041682 sec","job_status":"done","queue_duration_s":1.567054,"scheduling_latency_s":1.567091,"redis_calls":8,"redis_duration_s":0.0023899999999999998,"redis_read_bytes":133,"redis_write_bytes":963,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000308,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":137,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001565,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.041682,"completed_at":"2026-11-10T11:05:08.327Z","load_balancing_strategy":"primary","db_duration_s":0.010312,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 367.857165] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"028c356789c56517e1b9f8f0","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-028c356789c56517e1b9f8f0: start","job_status":"start","queue_duration_s":0.101213,"scheduling_latency_s":0.101259}1468gitlab # [ 367.870206] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"028c356789c56517e1b9f8f0","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-028c356789c56517e1b9f8f0: done: 0.033175 sec","job_status":"done","queue_duration_s":0.101213,"scheduling_latency_s":0.101259,"redis_calls":13,"redis_duration_s":0.003044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001171,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033175,"completed_at":"2026-11-10T11:05:08.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 367.911525] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"027d4c9198c7e578e569b976","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-027d4c9198c7e578e569b976: start","job_status":"start","queue_duration_s":0.136631,"scheduling_latency_s":0.136666}1470gitlab # [ 367.924820] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"027d4c9198c7e578e569b976","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-027d4c9198c7e578e569b976: done: 0.028859 sec","job_status":"done","queue_duration_s":0.136631,"scheduling_latency_s":0.136666,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001744,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028859,"completed_at":"2026-11-10T11:05:08.394Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 367.965540] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfdb1baedef34963c3ffd9ef","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfdb1baedef34963c3ffd9ef: start","job_status":"start","queue_duration_s":0.169776,"scheduling_latency_s":0.169812}1472gitlab # [ 367.978531] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfdb1baedef34963c3ffd9ef","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfdb1baedef34963c3ffd9ef: done: 0.030228 sec","job_status":"done","queue_duration_s":0.169776,"scheduling_latency_s":0.169812,"redis_calls":13,"redis_duration_s":0.003016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030228,"completed_at":"2026-11-10T11:05:08.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 368.020855] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"263220deba2853e59db544cf","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-263220deba2853e59db544cf: start","job_status":"start","queue_duration_s":0.202314,"scheduling_latency_s":0.20235}1474gitlab # [ 368.034123] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"263220deba2853e59db544cf","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-263220deba2853e59db544cf: done: 0.028605 sec","job_status":"done","queue_duration_s":0.202314,"scheduling_latency_s":0.20235,"redis_calls":13,"redis_duration_s":0.002882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028605,"completed_at":"2026-11-10T11:05:08.459Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 368.074943] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7b1c48aec47088b4a5706ba","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7b1c48aec47088b4a5706ba: start","job_status":"start","queue_duration_s":0.23311,"scheduling_latency_s":0.233146}1476gitlab # [ 368.088510] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7b1c48aec47088b4a5706ba","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7b1c48aec47088b4a5706ba: done: 0.0225 sec","job_status":"done","queue_duration_s":0.23311,"scheduling_latency_s":0.233146,"redis_calls":13,"redis_duration_s":0.001907,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001296,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0225,"completed_at":"2026-11-10T11:05:08.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 368.129234] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8391e78d1b2c1cae7ff6306b","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8391e78d1b2c1cae7ff6306b: start","job_status":"start","queue_duration_s":0.257304,"scheduling_latency_s":0.257334}1478gitlab # [ 368.142391] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8391e78d1b2c1cae7ff6306b","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8391e78d1b2c1cae7ff6306b: done: 0.022146 sec","job_status":"done","queue_duration_s":0.257304,"scheduling_latency_s":0.257334,"redis_calls":13,"redis_duration_s":0.001999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022146,"completed_at":"2026-11-10T11:05:08.508Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 368.183595] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ebe66b88e23acae32cc8672","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ebe66b88e23acae32cc8672: start","job_status":"start","queue_duration_s":0.281191,"scheduling_latency_s":0.28122}1480gitlab # [ 368.197901] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ebe66b88e23acae32cc8672","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ebe66b88e23acae32cc8672: done: 0.022091 sec","job_status":"done","queue_duration_s":0.281191,"scheduling_latency_s":0.28122,"redis_calls":13,"redis_duration_s":0.002155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022091,"completed_at":"2026-11-10T11:05:08.532Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 368.238762] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35fe9a222b18d3e99051ce8a","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35fe9a222b18d3e99051ce8a: start","job_status":"start","queue_duration_s":0.304885,"scheduling_latency_s":0.304913}1482gitlab # [ 368.251882] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35fe9a222b18d3e99051ce8a","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35fe9a222b18d3e99051ce8a: done: 0.021224 sec","job_status":"done","queue_duration_s":0.304885,"scheduling_latency_s":0.304913,"redis_calls":13,"redis_duration_s":0.002056,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001333,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021224,"completed_at":"2026-11-10T11:05:08.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 368.293577] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f1ef65bc0858cc9e1bd69be","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f1ef65bc0858cc9e1bd69be: start","job_status":"start","queue_duration_s":0.327768,"scheduling_latency_s":0.327799}1484gitlab # [ 368.306741] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f1ef65bc0858cc9e1bd69be","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f1ef65bc0858cc9e1bd69be: done: 0.021059 sec","job_status":"done","queue_duration_s":0.327768,"scheduling_latency_s":0.327799,"redis_calls":13,"redis_duration_s":0.002052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021059,"completed_at":"2026-11-10T11:05:08.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 368.348672] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f6ba243fcf65c0c5bcb096f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f6ba243fcf65c0c5bcb096f: start","job_status":"start","queue_duration_s":0.351833,"scheduling_latency_s":0.351863}1486gitlab # [ 368.361761] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f6ba243fcf65c0c5bcb096f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f6ba243fcf65c0c5bcb096f: done: 0.021645 sec","job_status":"done","queue_duration_s":0.351833,"scheduling_latency_s":0.351863,"redis_calls":13,"redis_duration_s":0.002035,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021645,"completed_at":"2026-11-10T11:05:08.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 368.403929] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba1e0c066b6870b4a2980623","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba1e0c066b6870b4a2980623: start","job_status":"start","queue_duration_s":0.375199,"scheduling_latency_s":0.375227}1488gitlab # [ 368.417179] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba1e0c066b6870b4a2980623","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba1e0c066b6870b4a2980623: done: 0.022304 sec","job_status":"done","queue_duration_s":0.375199,"scheduling_latency_s":0.375227,"redis_calls":13,"redis_duration_s":0.002656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001566,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022304,"completed_at":"2026-11-10T11:05:08.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 368.458183] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adfcd3e699952e975147a859","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adfcd3e699952e975147a859: start","job_status":"start","queue_duration_s":0.399174,"scheduling_latency_s":0.399203}1490gitlab # [ 368.471362] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adfcd3e699952e975147a859","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adfcd3e699952e975147a859: done: 0.02219 sec","job_status":"done","queue_duration_s":0.399174,"scheduling_latency_s":0.399203,"redis_calls":13,"redis_duration_s":0.002557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02219,"completed_at":"2026-11-10T11:05:08.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 368.512789] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39484a27ecd1f233655df285","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39484a27ecd1f233655df285: start","job_status":"start","queue_duration_s":0.42302,"scheduling_latency_s":0.423049}1492gitlab # [ 368.525785] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39484a27ecd1f233655df285","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39484a27ecd1f233655df285: done: 0.023238 sec","job_status":"done","queue_duration_s":0.42302,"scheduling_latency_s":0.423049,"redis_calls":13,"redis_duration_s":0.002619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000975,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023238,"completed_at":"2026-11-10T11:05:08.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 368.567106] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9738b71378c0a79c415deb7","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9738b71378c0a79c415deb7: start","job_status":"start","queue_duration_s":0.448082,"scheduling_latency_s":0.448111}1494gitlab # [ 368.580282] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9738b71378c0a79c415deb7","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9738b71378c0a79c415deb7: done: 0.023181 sec","job_status":"done","queue_duration_s":0.448082,"scheduling_latency_s":0.448111,"redis_calls":13,"redis_duration_s":0.002588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001689,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023181,"completed_at":"2026-11-10T11:05:08.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 368.622836] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45538c99d24932df4def559e","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45538c99d24932df4def559e: start","job_status":"start","queue_duration_s":0.472956,"scheduling_latency_s":0.472983}1496gitlab # [ 368.635897] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45538c99d24932df4def559e","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45538c99d24932df4def559e: done: 0.02153 sec","job_status":"done","queue_duration_s":0.472956,"scheduling_latency_s":0.472983,"redis_calls":13,"redis_duration_s":0.0025659999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001609,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02153,"completed_at":"2026-11-10T11:05:08.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 368.677622] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"916b1c9fd1682e85b361c74b","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-916b1c9fd1682e85b361c74b: start","job_status":"start","queue_duration_s":0.496246,"scheduling_latency_s":0.496282}1498gitlab # [ 368.690654] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"916b1c9fd1682e85b361c74b","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-916b1c9fd1682e85b361c74b: done: 0.022151 sec","job_status":"done","queue_duration_s":0.496246,"scheduling_latency_s":0.496282,"redis_calls":13,"redis_duration_s":0.0026690000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001775,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022151,"completed_at":"2026-11-10T11:05:08.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 368.732145] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a9f4fe591363faa40522b4","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a9f4fe591363faa40522b4: start","job_status":"start","queue_duration_s":0.521822,"scheduling_latency_s":0.521851}1500gitlab # [ 368.745014] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a9f4fe591363faa40522b4","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a9f4fe591363faa40522b4: done: 0.022304 sec","job_status":"done","queue_duration_s":0.521822,"scheduling_latency_s":0.521851,"redis_calls":13,"redis_duration_s":0.003109,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001679,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022304,"completed_at":"2026-11-10T11:05:08.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 368.786038] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3193c901129d291942aefd37","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3193c901129d291942aefd37: start","job_status":"start","queue_duration_s":0.545956,"scheduling_latency_s":0.545985}1502gitlab # [ 368.798777] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3193c901129d291942aefd37","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3193c901129d291942aefd37: done: 0.023142 sec","job_status":"done","queue_duration_s":0.545956,"scheduling_latency_s":0.545985,"redis_calls":13,"redis_duration_s":0.004169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023142,"completed_at":"2026-11-10T11:05:08.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 368.841262] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99f01d76effc58ddfd4a1d41","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99f01d76effc58ddfd4a1d41: start","job_status":"start","queue_duration_s":0.570723,"scheduling_latency_s":0.570747}1504gitlab # [ 368.854341] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99f01d76effc58ddfd4a1d41","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99f01d76effc58ddfd4a1d41: done: 0.022087 sec","job_status":"done","queue_duration_s":0.570723,"scheduling_latency_s":0.570747,"redis_calls":13,"redis_duration_s":0.003068,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00157,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001498,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022087,"completed_at":"2026-11-10T11:05:08.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 368.895909] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3175e2b0261aa2460a9728b5","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3175e2b0261aa2460a9728b5: start","job_status":"start","queue_duration_s":0.594874,"scheduling_latency_s":0.5949}1506gitlab # [ 368.909064] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3175e2b0261aa2460a9728b5","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3175e2b0261aa2460a9728b5: done: 0.036991 sec","job_status":"done","queue_duration_s":0.594874,"scheduling_latency_s":0.5949,"redis_calls":13,"redis_duration_s":0.011657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036991,"completed_at":"2026-11-10T11:05:08.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 368.950150] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c94eaffe5dc72ab0d8dcb3","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c94eaffe5dc72ab0d8dcb3: start","job_status":"start","queue_duration_s":0.635532,"scheduling_latency_s":0.635565}1508gitlab # [ 368.963261] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c94eaffe5dc72ab0d8dcb3","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c94eaffe5dc72ab0d8dcb3: done: 0.02615 sec","job_status":"done","queue_duration_s":0.635532,"scheduling_latency_s":0.635565,"redis_calls":13,"redis_duration_s":0.003131,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02615,"completed_at":"2026-11-10T11:05:08.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 369.004684] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bae0bbbb7f934982be8e1b8","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bae0bbbb7f934982be8e1b8: start","job_status":"start","queue_duration_s":0.663761,"scheduling_latency_s":0.663794}1510gitlab # [ 369.018839] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bae0bbbb7f934982be8e1b8","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bae0bbbb7f934982be8e1b8: done: 0.02561 sec","job_status":"done","queue_duration_s":0.663761,"scheduling_latency_s":0.663794,"redis_calls":13,"redis_duration_s":0.005606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02561,"completed_at":"2026-11-10T11:05:08.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 369.060051] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55b1e7142c69279e46ecfec1","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55b1e7142c69279e46ecfec1: start","job_status":"start","queue_duration_s":0.691754,"scheduling_latency_s":0.691786}1512gitlab # [ 369.072892] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55b1e7142c69279e46ecfec1","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55b1e7142c69279e46ecfec1: done: 0.025306 sec","job_status":"done","queue_duration_s":0.691754,"scheduling_latency_s":0.691786,"redis_calls":13,"redis_duration_s":0.002915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025306,"completed_at":"2026-11-10T11:05:08.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 369.113558] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7e36ac7dba7c868efa7e10b","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7e36ac7dba7c868efa7e10b: start","job_status":"start","queue_duration_s":0.72171,"scheduling_latency_s":0.721744}1514gitlab # [ 369.126540] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7e36ac7dba7c868efa7e10b","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7e36ac7dba7c868efa7e10b: done: 0.028772 sec","job_status":"done","queue_duration_s":0.72171,"scheduling_latency_s":0.721744,"redis_calls":13,"redis_duration_s":0.003467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001281,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028772,"completed_at":"2026-11-10T11:05:08.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 369.167669] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:08.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fb61b5d2efb5b3728797893","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fb61b5d2efb5b3728797893: start","job_status":"start","queue_duration_s":0.753285,"scheduling_latency_s":0.753318}1516gitlab # [ 369.180790] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fb61b5d2efb5b3728797893","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fb61b5d2efb5b3728797893: done: 0.026868 sec","job_status":"done","queue_duration_s":0.753285,"scheduling_latency_s":0.753318,"redis_calls":13,"redis_duration_s":0.003704,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001073,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002631,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026868,"completed_at":"2026-11-10T11:05:09.009Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 369.222688] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"616f346019d048bfa1106379","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-616f346019d048bfa1106379: start","job_status":"start","queue_duration_s":0.782419,"scheduling_latency_s":0.782453}1518gitlab # [ 369.235806] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"616f346019d048bfa1106379","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-616f346019d048bfa1106379: done: 0.024134 sec","job_status":"done","queue_duration_s":0.782419,"scheduling_latency_s":0.782453,"redis_calls":13,"redis_duration_s":0.002987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024134,"completed_at":"2026-11-10T11:05:09.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 369.277283] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e352ecede7a34728b510c0a","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e352ecede7a34728b510c0a: start","job_status":"start","queue_duration_s":0.808601,"scheduling_latency_s":0.808633}1520gitlab # [ 369.290250] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e352ecede7a34728b510c0a","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e352ecede7a34728b510c0a: done: 0.025414 sec","job_status":"done","queue_duration_s":0.808601,"scheduling_latency_s":0.808633,"redis_calls":13,"redis_duration_s":0.002565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025414,"completed_at":"2026-11-10T11:05:09.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 369.331473] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dac163961039b78f0b3a0410","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dac163961039b78f0b3a0410: start","job_status":"start","queue_duration_s":0.836081,"scheduling_latency_s":0.836114}1522gitlab # [ 369.344632] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dac163961039b78f0b3a0410","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dac163961039b78f0b3a0410: done: 0.022826 sec","job_status":"done","queue_duration_s":0.836081,"scheduling_latency_s":0.836114,"redis_calls":13,"redis_duration_s":0.0028079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001023,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001785,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022826,"completed_at":"2026-11-10T11:05:09.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 369.386729] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6c2e5707e76a0851ecd1c4f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6c2e5707e76a0851ecd1c4f: start","job_status":"start","queue_duration_s":0.860859,"scheduling_latency_s":0.860911}1524gitlab # [ 369.399980] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6c2e5707e76a0851ecd1c4f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6c2e5707e76a0851ecd1c4f: done: 0.022162 sec","job_status":"done","queue_duration_s":0.860859,"scheduling_latency_s":0.860911,"redis_calls":13,"redis_duration_s":0.002543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001652,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022162,"completed_at":"2026-11-10T11:05:09.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 369.442686] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e502c2c826882e249ef70e5e","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e502c2c826882e249ef70e5e: start","job_status":"start","queue_duration_s":0.884765,"scheduling_latency_s":0.884798}1526gitlab # [ 369.455903] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e502c2c826882e249ef70e5e","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e502c2c826882e249ef70e5e: done: 0.021879 sec","job_status":"done","queue_duration_s":0.884765,"scheduling_latency_s":0.884798,"redis_calls":13,"redis_duration_s":0.002509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001656,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021879,"completed_at":"2026-11-10T11:05:09.135Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 369.497412] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31c2a27be522c02bf49bc592","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31c2a27be522c02bf49bc592: start","job_status":"start","queue_duration_s":0.909368,"scheduling_latency_s":0.909398}1528gitlab # [ 369.510428] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31c2a27be522c02bf49bc592","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31c2a27be522c02bf49bc592: done: 0.021214 sec","job_status":"done","queue_duration_s":0.909368,"scheduling_latency_s":0.909398,"redis_calls":13,"redis_duration_s":0.002565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001632,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021214,"completed_at":"2026-11-10T11:05:09.159Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 369.551556] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"885392627ea200bb50517525","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-885392627ea200bb50517525: start","job_status":"start","queue_duration_s":0.932415,"scheduling_latency_s":0.932443}1530gitlab # [ 369.564681] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"885392627ea200bb50517525","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-885392627ea200bb50517525: done: 0.021336 sec","job_status":"done","queue_duration_s":0.932415,"scheduling_latency_s":0.932443,"redis_calls":13,"redis_duration_s":0.0026,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021336,"completed_at":"2026-11-10T11:05:09.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 369.605631] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7d97bb9d5c93aeef5fd7b0f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7d97bb9d5c93aeef5fd7b0f: start","job_status":"start","queue_duration_s":0.955459,"scheduling_latency_s":0.955488}1532gitlab # [ 369.618652] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7d97bb9d5c93aeef5fd7b0f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7d97bb9d5c93aeef5fd7b0f: done: 0.021566 sec","job_status":"done","queue_duration_s":0.955459,"scheduling_latency_s":0.955488,"redis_calls":13,"redis_duration_s":0.00263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001739,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021566,"completed_at":"2026-11-10T11:05:09.206Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 369.661651] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f8da98badfdf3944d98c7e9","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f8da98badfdf3944d98c7e9: start","job_status":"start","queue_duration_s":0.979732,"scheduling_latency_s":0.979763}1534gitlab # [ 369.682363] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f8da98badfdf3944d98c7e9","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f8da98badfdf3944d98c7e9: done: 0.02147 sec","job_status":"done","queue_duration_s":0.979732,"scheduling_latency_s":0.979763,"redis_calls":13,"redis_duration_s":0.002536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001617,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02147,"completed_at":"2026-11-10T11:05:09.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 369.742891] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8185bbefb66e047b1479ee3c","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8185bbefb66e047b1479ee3c: start","job_status":"start","queue_duration_s":1.003169,"scheduling_latency_s":1.003197}1536gitlab # [ 369.757958] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8185bbefb66e047b1479ee3c","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8185bbefb66e047b1479ee3c: done: 0.021637 sec","job_status":"done","queue_duration_s":1.003169,"scheduling_latency_s":1.003197,"redis_calls":13,"redis_duration_s":0.002747,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001797,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021637,"completed_at":"2026-11-10T11:05:09.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 369.800139] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dd2b6632b8249f664f1a284","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dd2b6632b8249f664f1a284: start","job_status":"start","queue_duration_s":1.026556,"scheduling_latency_s":1.026586}1538gitlab # [ 369.813258] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dd2b6632b8249f664f1a284","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dd2b6632b8249f664f1a284: done: 0.025068 sec","job_status":"done","queue_duration_s":1.026556,"scheduling_latency_s":1.026586,"redis_calls":13,"redis_duration_s":0.002746,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001834,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025068,"completed_at":"2026-11-10T11:05:09.280Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 369.856757] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bfd354e366d33a3773ba7a7","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bfd354e366d33a3773ba7a7: start","job_status":"start","queue_duration_s":1.053282,"scheduling_latency_s":1.053308}1540gitlab # [ 369.877651] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bfd354e366d33a3773ba7a7","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bfd354e366d33a3773ba7a7: done: 0.021419 sec","job_status":"done","queue_duration_s":1.053282,"scheduling_latency_s":1.053308,"redis_calls":13,"redis_duration_s":0.0023350000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001531,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021419,"completed_at":"2026-11-10T11:05:09.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 369.936903] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09103be8dba40f348d6e4ac7","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09103be8dba40f348d6e4ac7: start","job_status":"start","queue_duration_s":1.076399,"scheduling_latency_s":1.076423}1542gitlab # [ 369.951230] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09103be8dba40f348d6e4ac7","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09103be8dba40f348d6e4ac7: done: 0.019249 sec","job_status":"done","queue_duration_s":1.076399,"scheduling_latency_s":1.076423,"redis_calls":13,"redis_duration_s":0.002157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001413,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019249,"completed_at":"2026-11-10T11:05:09.324Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 369.993144] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c39b702f0a4c868691a9816","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c39b702f0a4c868691a9816: start","job_status":"start","queue_duration_s":1.097383,"scheduling_latency_s":1.097409}1544gitlab # [ 370.006196] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c39b702f0a4c868691a9816","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c39b702f0a4c868691a9816: done: 0.022684 sec","job_status":"done","queue_duration_s":1.097383,"scheduling_latency_s":1.097409,"redis_calls":13,"redis_duration_s":0.004902999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022684,"completed_at":"2026-11-10T11:05:09.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 370.048746] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3738df26926387f4f977d9b0","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3738df26926387f4f977d9b0: start","job_status":"start","queue_duration_s":1.12211,"scheduling_latency_s":1.122141}1546gitlab # [ 370.061674] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3738df26926387f4f977d9b0","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3738df26926387f4f977d9b0: done: 0.021022 sec","job_status":"done","queue_duration_s":1.12211,"scheduling_latency_s":1.122141,"redis_calls":13,"redis_duration_s":0.00275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001797,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021022,"completed_at":"2026-11-10T11:05:09.372Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 370.102832] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d12ac99a3378f264f53992","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d12ac99a3378f264f53992: start","job_status":"start","queue_duration_s":1.145972,"scheduling_latency_s":1.146001}1548gitlab # [ 370.115852] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d12ac99a3378f264f53992","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d12ac99a3378f264f53992: done: 0.020702 sec","job_status":"done","queue_duration_s":1.145972,"scheduling_latency_s":1.146001,"redis_calls":13,"redis_duration_s":0.002477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001587,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020702,"completed_at":"2026-11-10T11:05:09.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 370.157302] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10ddbab0955e8974d0a3e1f9","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10ddbab0955e8974d0a3e1f9: start","job_status":"start","queue_duration_s":1.168319,"scheduling_latency_s":1.168344}1550gitlab # [ 370.170391] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10ddbab0955e8974d0a3e1f9","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10ddbab0955e8974d0a3e1f9: done: 0.020301 sec","job_status":"done","queue_duration_s":1.168319,"scheduling_latency_s":1.168344,"redis_calls":13,"redis_duration_s":0.002386,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001464,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020301,"completed_at":"2026-11-10T11:05:09.417Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 370.211585] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84ed463fa9d3847691a729bf","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84ed463fa9d3847691a729bf: start","job_status":"start","queue_duration_s":1.190337,"scheduling_latency_s":1.190378}1552gitlab # [ 370.224681] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84ed463fa9d3847691a729bf","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84ed463fa9d3847691a729bf: done: 0.022172 sec","job_status":"done","queue_duration_s":1.190337,"scheduling_latency_s":1.190378,"redis_calls":13,"redis_duration_s":0.0033049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001359,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022172,"completed_at":"2026-11-10T11:05:09.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 370.267376] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88a8de05a0539a226685da1f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88a8de05a0539a226685da1f: start","job_status":"start","queue_duration_s":1.214242,"scheduling_latency_s":1.214272}1554gitlab # [ 370.287885] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88a8de05a0539a226685da1f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88a8de05a0539a226685da1f: done: 0.022706 sec","job_status":"done","queue_duration_s":1.214242,"scheduling_latency_s":1.214272,"redis_calls":13,"redis_duration_s":0.002502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001574,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022706,"completed_at":"2026-11-10T11:05:09.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 370.347534] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c609ab8009b696d09236fed","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c609ab8009b696d09236fed: start","job_status":"start","queue_duration_s":1.238742,"scheduling_latency_s":1.23877}1556gitlab # [ 370.362755] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c609ab8009b696d09236fed","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c609ab8009b696d09236fed: done: 0.022412 sec","job_status":"done","queue_duration_s":1.238742,"scheduling_latency_s":1.23877,"redis_calls":13,"redis_duration_s":0.002745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022412,"completed_at":"2026-11-10T11:05:09.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 370.409678] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98a101ad3314efd0b7733086","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98a101ad3314efd0b7733086: start","job_status":"start","queue_duration_s":1.263152,"scheduling_latency_s":1.263185}1558gitlab # [ 370.423044] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98a101ad3314efd0b7733086","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98a101ad3314efd0b7733086: done: 0.022999 sec","job_status":"done","queue_duration_s":1.263152,"scheduling_latency_s":1.263185,"redis_calls":13,"redis_duration_s":0.0031450000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001801,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022999,"completed_at":"2026-11-10T11:05:09.515Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559gitlab # [ 370.465525] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef1b7c2bbc34d5dcf9278ddc","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef1b7c2bbc34d5dcf9278ddc: start","job_status":"start","queue_duration_s":1.287971,"scheduling_latency_s":1.287999}1560gitlab # [ 370.478592] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef1b7c2bbc34d5dcf9278ddc","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef1b7c2bbc34d5dcf9278ddc: done: 0.023534 sec","job_status":"done","queue_duration_s":1.287971,"scheduling_latency_s":1.287999,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023534,"completed_at":"2026-11-10T11:05:09.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 370.520638] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2da279e7940577d520249834","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2da279e7940577d520249834: start","job_status":"start","queue_duration_s":1.314656,"scheduling_latency_s":1.314683}1562gitlab # [ 370.533610] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2da279e7940577d520249834","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2da279e7940577d520249834: done: 0.022778 sec","job_status":"done","queue_duration_s":1.314656,"scheduling_latency_s":1.314683,"redis_calls":13,"redis_duration_s":0.002872,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022778,"completed_at":"2026-11-10T11:05:09.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 370.574865] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b628bb3162dbb52b8a2379c","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b628bb3162dbb52b8a2379c: start","job_status":"start","queue_duration_s":1.339147,"scheduling_latency_s":1.339175}1564gitlab # [ 370.587939] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b628bb3162dbb52b8a2379c","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b628bb3162dbb52b8a2379c: done: 0.022066 sec","job_status":"done","queue_duration_s":1.339147,"scheduling_latency_s":1.339175,"redis_calls":13,"redis_duration_s":0.002648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022066,"completed_at":"2026-11-10T11:05:09.590Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565gitlab # [ 370.629998] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b966c52044b507514556342f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b966c52044b507514556342f: start","job_status":"start","queue_duration_s":1.363065,"scheduling_latency_s":1.363094}1566gitlab # [ 370.643062] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:05:09.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b966c52044b507514556342f","created_at":"2026-11-10T11:05:08.078Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-838d72e3c61d4b8c","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"833e65eab47007ba1c963e823fc97a3f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:05:08.229Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b966c52044b507514556342f: done: 0.021046 sec","job_status":"done","queue_duration_s":1.363065,"scheduling_latency_s":1.363094,"redis_calls":13,"redis_duration_s":0.0025989999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001718,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021046,"completed_at":"2026-11-10T11:05:09.613Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 419.570807] postgres[4060]: [4060] ERROR: relation "table_without_models" does not exist1568gitlab # [ 419.571300] postgres[4060]: [4060] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1569gitlab # [ 424.698476] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:05.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"0a14c3d89e35967639cbd898","created_at":"2026-11-10T11:06:05.234Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.246Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-0a14c3d89e35967639cbd898: start","job_status":"start","queue_duration_s":0.002728,"scheduling_latency_s":0.002777}1570gitlab # [ 424.873813] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:05.424Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"0a14c3d89e35967639cbd898","created_at":"2026-11-10T11:06:05.234Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.246Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-0a14c3d89e35967639cbd898: done: 0.175605 sec","job_status":"done","queue_duration_s":0.002728,"scheduling_latency_s":0.002777,"redis_calls":14,"redis_duration_s":0.051197,"redis_read_bytes":15,"redis_write_bytes":4022,"redis_queues_calls":2,"redis_queues_duration_s":0.001688,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.043617,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.005892,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.175605,"completed_at":"2026-11-10T11:06:05.424Z","load_balancing_strategy":"primary","db_duration_s":0.005759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571gitlab # [ 424.939346] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:05.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"bbd26fb767b4840eef7dc0ff","created_at":"2026-11-10T11:06:05.266Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e82d6fb12d4195b39f6683dccc32fe2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.283Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-bbd26fb767b4840eef7dc0ff: start","job_status":"start","queue_duration_s":0.146038,"scheduling_latency_s":0.146083}1572gitlab # [ 425.892537] runuser[3572]: pam_unix(runuser:session): session closed for user gitlab1573gitlab # [ 425.893642] setup-gitlab-start[3536]: + systemctl restart gitlab-sidekiq.service1574gitlab # [ 425.939541] systemd[1]: Stopping gitlab-sidekiq.service...1575gitlab # [ 425.944117] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:06.495Z","message":"Shutting down"}1576gitlab # [ 425.946441] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:06.497Z","message":"Scheduler exiting..."}1577gitlab # [ 425.948857] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:06.498Z","message":"Terminating quiet threads for default capsule"}1578gitlab # [ 425.951576] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:06.498Z","message":"Scheduler exiting..."}1579gitlab # [ 426.002403] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:06.553Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"bbd26fb767b4840eef7dc0ff","created_at":"2026-11-10T11:06:05.266Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e82d6fb12d4195b39f6683dccc32fe2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.283Z","job_size_bytes":2,"pid":2273,"sidekiq_tid":"m7q5","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-bbd26fb767b4840eef7dc0ff: done: 1.124224 sec","job_status":"done","queue_duration_s":0.146038,"scheduling_latency_s":0.146083,"redis_calls":12,"redis_duration_s":0.008976,"redis_read_bytes":137,"redis_write_bytes":1339,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00106,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":352,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002579,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005337,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.238,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.238,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.537946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00288804,"exclusive_lock_hold_duration_s":1.02095572,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":1.124224,"completed_at":"2026-11-10T11:06:06.553Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.574121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580gitlab # [ 426.450874] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:07.002Z","message":"Bye!"}1581gitlab # [ 426.481559] sidekiq-cluster[2273]: {"severity":"INFO","time":"2026-11-10T11:06:07.033Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2273,"worker_id":"sidekiq_0","memwd_rss_bytes":931233792,"retry":0}1582gitlab # [ 427.255771] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1583gitlab # [ 427.256133] systemd[1]: Stopped gitlab-sidekiq.service.1584gitlab # [ 427.256268] systemd[1]: gitlab-sidekiq.service: Consumed 2min 41.003s CPU time over 4min 38.117s wall clock time, 946.1M memory peak, 4.4M read from disk.1585gitlab # [ 427.263952] systemd[1]: Started gitlab-sidekiq.service.1586gitlab # [ 427.267350] setup-gitlab-start[3536]: + mkdir -p /var/lib/secrets1587gitlab # [ 427.270551] setup-gitlab-start[3536]: + echo nixbot-test-token-of-20-chars1588gitlab # [ 427.272099] setup-gitlab-start[3536]: + chmod 644 /var/lib/secrets/gitlab-token1589gitlab # [ 427.274538] setup-gitlab-start[3536]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1590gitlab # [ 427.781991] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:08 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 4771591gitlab # [ 428.707343] gitaly[923]: time="2026-11-10T11:06:09.258Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.257" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1592gitlab # [ 428.716483] gitaly[923]: time="2026-11-10T11:06:09.258Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.257" grpc.time_ms=0.476 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593gitlab # [ 428.951181] gitaly[923]: time="2026-11-10T11:06:09.503Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.252" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.502" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594gitlab # [ 428.987177] gitaly[923]: time="2026-11-10T11:06:09.538Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.252" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.502" grpc.time_ms=35.326 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595gitlab # [ 429.053859] gitaly[923]: time="2026-11-10T11:06:09.605Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 diskcache=ac180ebb-c8af-444d-9436-6b67b80b46b2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T11:06:19.001" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.551" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596gitlab # [ 429.071278] gitaly[923]: time="2026-11-10T11:06:09.606Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=26 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=31 command.spawn_token_fork_ms=4 command.system_time_ms=20 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T11:06:19.001" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.551" grpc.time_ms=54.159 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1597gitlab # [ 429.092810] gitaly[923]: time="2026-11-10T11:06:09.627Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.076" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.626" pid=923 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1598gitlab # [ 429.106720] gitaly[923]: time="2026-11-10T11:06:09.627Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.076" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.626" grpc.time_ms=0.996 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1599gitlab # [ 429.120250] gitaly[923]: time="2026-11-10T11:06:09.649Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:06:19.085" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.636" grpc.time_ms=13.132 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1600gitlab # [ 429.225358] gitaly[923]: time="2026-11-10T11:06:09.777Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.226" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.777" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1601gitlab # [ 429.239582] gitaly[923]: time="2026-11-10T11:06:09.777Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.226" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.777" grpc.time_ms=0.494 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602gitlab # [ 429.283263] gitaly[923]: time="2026-11-10T11:06:09.835Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 diskcache=ae5f994b-ef3c-4047-a3b8-47df9f47231b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T11:06:19.233" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.784" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 429.298086] gitaly[923]: time="2026-11-10T11:06:09.835Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=25 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1436 command.oublock=32 command.real_time_ms=29 command.spawn_token_fork_ms=5 command.system_time_ms=25 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T11:06:19.233" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.784" grpc.time_ms=51.437 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604gitlab # [ 429.314792] gitaly[923]: time="2026-11-10T11:06:09.847Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.846" pid=923 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1605gitlab # [ 429.326543] gitaly[923]: time="2026-11-10T11:06:09.847Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.846" grpc.time_ms=0.928 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1606gitlab # [ 429.338437] gitaly[923]: time="2026-11-10T11:06:09.867Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:06:19.013" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.853" grpc.time_ms=12.881 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 429.369453] gitaly[923]: time="2026-11-10T11:06:09.921Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:06:19.080" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.921" grpc.time_ms=0.283 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1608gitlab # [ 429.382980] gitaly[923]: time="2026-11-10T11:06:09.933Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T11:06:19.092" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:09.932" grpc.time_ms=0.407 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1609gitlab # [ 430.150609] gitaly[923]: time="2026-11-10T11:06:10.701Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J60JXBF04HJ5XKB8Z8RQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:06:20.139" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:10.689" grpc.time_ms=11.904 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 430.906079] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:11 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 31081611gitlab # [ 430.909173] setup-gitlab-start[4123]: {"id":1,"message":null,"error":null}1612gitlab # [ 435.153462] sidekiq-cluster[4110]: {"severity":"INFO","time":"2026-11-10T11:06:15.642Z","message":"Starting cluster with 1 processes"}1613gitlab # [ 436.088428] gitaly[923]: time="2026-11-10T11:06:16.639Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J68H8S0045V7RN7FWQD5M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:06:26.076" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:16.626" grpc.time_ms=12.732 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 436.230811] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:16 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 2931615gitlab # [ 436.235406] setup-gitlab-start[3536]: + mkdir -p /tmp/test-flake1616gitlab # [ 436.239055] setup-gitlab-start[3536]: + cd /tmp/test-flake1617gitlab # [ 436.240324] setup-gitlab-start[3536]: + git init -b master1618gitlab # [ 436.271315] setup-gitlab-start[4200]: Initialized empty Git repository in /tmp/test-flake/.git/1619gitlab # [ 436.273617] setup-gitlab-start[3536]: + git config user.name test1620gitlab # [ 436.277426] setup-gitlab-start[3536]: + git config user.email test@example.com1621gitlab # [ 436.282195] setup-gitlab-start[3536]: + cat1622gitlab # [ 436.286026] setup-gitlab-start[3536]: + git add flake.nix1623gitlab # [ 436.291662] setup-gitlab-start[3536]: + git commit -m 'initial commit'1624gitlab # [ 436.316097] setup-gitlab-start[4205]: [master (root-commit) 5599f99] initial commit1625gitlab # [ 436.318004] setup-gitlab-start[4205]: 1 file changed, 10 insertions(+)1626gitlab # [ 436.319482] setup-gitlab-start[4205]: create mode 100644 flake.nix1627gitlab # [ 436.321543] setup-gitlab-start[3536]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1628gitlab # [ 436.325207] setup-gitlab-start[3536]: + git push -u origin master1629gitlab # [ 436.610316] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 2491630gitlab # [ 437.487621] gitaly[923]: time="2026-11-10T11:06:18.039Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J696B10S73G7X54T7CFGV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:18.026" grpc.time_ms=12.077 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1631gitlab # [ 437.488431] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:18 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 8751632gitlab # [ 437.828781] gitaly[923]: time="2026-11-10T11:06:18.380Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:06:28.115" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:18.366" grpc.time_ms=14.238 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1633gitlab # [ 437.897571] gitaly[923]: time="2026-11-10T11:06:18.449Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T11:06:48.035" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T11:06:18.436" grpc.time_ms=13.338 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1634gitlab # [ 437.938061] gitaly[923]: time="2026-11-10T11:06:18.490Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ duration_ms=246ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/allowed"1635gitlab # [ 437.994465] gitaly[923]: time="2026-11-10T11:06:18.546Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ duration_ms=55ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/pre_receive"1636gitlab # [ 437.999645] gitaly[923]: time="2026-11-10T11:06:18.547Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5315 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:06:18.241" grpc.time_ms=305.146 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1637gitlab # [ 438.037511] gitaly[923]: time="2026-11-10T11:06:18.589Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:06:18.588" grpc.time_ms=1.094 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1638gitlab # [ 438.094617] gitaly[923]: time="2026-11-10T11:06:18.646Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:06:18.644" grpc.time_ms=1.29 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1639gitlab # [ 438.146628] gitaly[923]: time="2026-11-10T11:06:18.698Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:06:18.697" grpc.time_ms=1.136 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1640gitlab # [ 438.293874] gitaly[923]: time="2026-11-10T11:06:18.845Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:06:28.283" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:18.833" grpc.time_ms=11.995 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1641gitlab # [ 438.360939] gitaly[923]: time="2026-11-10T11:06:18.912Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=0 command.system_time_ms=17 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T11:06:28.042" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:18.883" grpc.time_ms=29.232 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1642gitlab # [ 438.383575] gitaly[923]: time="2026-11-10T11:06:18.930Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ duration_ms=186ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/post_receive"1643gitlab # [ 438.388564] gitaly[923]: time="2026-11-10T11:06:18.931Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4809 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:06:18.742" grpc.time_ms=188.072 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1644gitlab # [ 438.402519] gitaly[923]: time="2026-11-10T11:06:18.944Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ diskcache=457a7484-65cc-4875-a8c5-f5da2b5737fe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:18.149" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1645gitlab # [ 438.417317] setup-gitlab-start[4209]: To http://gitlab/root/test-flake.git1646gitlab # [ 438.418505] setup-gitlab-start[4209]: * [new branch] master -> master1647gitlab # [ 438.419629] setup-gitlab-start[4209]: branch 'master' set up to track 'origin/master'.1648gitlab # [ 438.421014] gitlab-workhorse[829]: time="2026-11-10T11:06:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521649gitlab # [ 438.423301] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:18 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 8941650gitlab # [ 438.426178] systemd[1]: Finished setup-gitlab.service.1651gitlab # [ 438.427092] gitaly[923]: time="2026-11-10T11:06:18.945Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=245 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13376 command.oublock=32 command.real_time_ms=786 command.spawn_token_fork_ms=1 command.system_time_ms=149 command.user_time_ms=96 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:18.149" grpc.time_ms=795.601 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1652gitlab # [ 438.443017] systemd[1]: Starting nixbot CI service...1653gitlab: (finished: waiting for unit setup-gitlab.service, in 439.16 seconds)1654gitlab: waiting for unit nixbot.service1655gitlab # [ 440.642647] nixbot[4291]: {"ts": "2026-11-10T11:06:21.193Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1656gitlab # [ 440.695212] nixbot[4291]: {"ts": "2026-11-10T11:06:21.247Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1657gitlab # [ 440.730545] nixbot[4291]: {"ts": "2026-11-10T11:06:21.282Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1658gitlab # [ 440.738194] nixbot[4291]: {"ts": "2026-11-10T11:06:21.289Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1659gitlab # [ 440.745744] nixbot[4291]: {"ts": "2026-11-10T11:06:21.297Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1660gitlab # [ 440.758094] nixbot[4291]: {"ts": "2026-11-10T11:06:21.309Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1661gitlab # [ 440.764093] nixbot[4291]: {"ts": "2026-11-10T11:06:21.316Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1662gitlab # [ 440.767711] nixbot[4291]: {"ts": "2026-11-10T11:06:21.318Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1663gitlab # [ 440.776508] nixbot[4291]: {"ts": "2026-11-10T11:06:21.328Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1664gitlab # [ 440.780444] nixbot[4291]: {"ts": "2026-11-10T11:06:21.331Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1665gitlab # [ 440.788377] nixbot[4291]: {"ts": "2026-11-10T11:06:21.340Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1666gitlab # [ 440.798673] nixbot[4291]: {"ts": "2026-11-10T11:06:21.350Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1667gitlab # [ 440.803290] nixbot[4291]: {"ts": "2026-11-10T11:06:21.355Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1668gitlab # [ 440.807110] nixbot[4291]: {"ts": "2026-11-10T11:06:21.357Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1669gitlab # [ 440.810937] nixbot[4291]: {"ts": "2026-11-10T11:06:21.362Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1670gitlab # [ 440.814781] nixbot[4291]: {"ts": "2026-11-10T11:06:21.366Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1671gitlab # [ 440.818539] nixbot[4291]: {"ts": "2026-11-10T11:06:21.368Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1672gitlab # [ 440.822370] nixbot[4291]: {"ts": "2026-11-10T11:06:21.371Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1673gitlab # [ 440.825904] nixbot[4291]: {"ts": "2026-11-10T11:06:21.377Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1674gitlab # [ 440.829377] nixbot[4291]: {"ts": "2026-11-10T11:06:21.380Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1675gitlab # [ 440.833015] nixbot[4291]: {"ts": "2026-11-10T11:06:21.382Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1676gitlab # [ 440.838264] nixbot[4291]: {"ts": "2026-11-10T11:06:21.390Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1677gitlab # [ 440.841982] nixbot[4291]: {"ts": "2026-11-10T11:06:21.392Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1678gitlab # [ 440.845714] nixbot[4291]: {"ts": "2026-11-10T11:06:21.394Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1679gitlab # [ 440.849237] nixbot[4291]: {"ts": "2026-11-10T11:06:21.396Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1680gitlab # [ 440.852674] nixbot[4291]: {"ts": "2026-11-10T11:06:21.399Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1681gitlab # [ 440.856108] nixbot[4291]: {"ts": "2026-11-10T11:06:21.404Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1682gitlab # [ 440.859345] nixbot[4291]: {"ts": "2026-11-10T11:06:21.407Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1683gitlab # [ 440.862563] nixbot[4291]: {"ts": "2026-11-10T11:06:21.410Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1684gitlab # [ 440.879670] nixbot[4291]: {"ts": "2026-11-10T11:06:21.431Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1685gitlab # [ 440.886267] nixbot[4291]: {"ts": "2026-11-10T11:06:21.438Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1686gitlab # [ 440.889743] nixbot[4291]: {"ts": "2026-11-10T11:06:21.441Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1687gitlab # [ 440.893338] nixbot[4291]: {"ts": "2026-11-10T11:06:21.443Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1688gitlab # [ 440.897744] nixbot[4291]: {"ts": "2026-11-10T11:06:21.449Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1689gitlab # [ 440.901371] nixbot[4291]: {"ts": "2026-11-10T11:06:21.451Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1690gitlab # [ 440.908415] nixbot[4291]: {"ts": "2026-11-10T11:06:21.460Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1691gitlab # [ 440.912520] nixbot[4291]: {"ts": "2026-11-10T11:06:21.464Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1692gitlab # [ 440.918310] nixbot[4291]: {"ts": "2026-11-10T11:06:21.470Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1693gitlab # [ 440.921937] nixbot[4291]: {"ts": "2026-11-10T11:06:21.473Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"}1694gitlab # [ 441.524897] nixbot[4291]: {"ts": "2026-11-10T11:06:22.076Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [4291]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1695gitlab # [ 441.525937] nixbot[4291]: {"ts": "2026-11-10T11:06:22.077Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1696gitlab # [ 441.578672] nixbot[4291]: {"ts": "2026-11-10T11:06:22.130Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:55730 - \"GET /health HTTP/1.1\" 200"}1697gitlab # [ 441.592465] systemd[1]: Started nixbot CI service.1698gitlab # [ 441.596371] systemd[1]: Reached target Multi-User System.1699gitlab # [ 441.597870] systemd[1]: Startup finished in 1.050s (kernel) + 4.830s (initrd) + 7min 15.716s (userspace) = 7min 21.597s.1700gitlab: (finished: waiting for unit nixbot.service, in 3.14 seconds)1701gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1702gitlab # [ 441.824678] nixbot[4291]: {"ts": "2026-11-10T11:06:22.373Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38412 - \"GET /health HTTP/1.1\" 200"}1703gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1704subtest: project discovered and webhook registered1705gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1706gitlab # [ 442.033552] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:22 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 4581707gitlab # [ 442.065539] nixbot[4291]: {"ts": "2026-11-10T11:06:22.617Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1708gitlab # [ 442.105304] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 2361709gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.28 seconds)1710gitlab # [ 442.173117] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 921711gitlab # [ 442.181870] nixbot[4291]: {"ts": "2026-11-10T11:06:22.733Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1712gitlab # [ 442.555558] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:23 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 3651713gitlab # [ 442.839726] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:23 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2701714gitlab # [ 442.991774] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 1381715gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1716gitlab # [ 443.124933] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:23 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 1221717gitlab # [ 443.252987] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 991718gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.15 seconds)1719(finished: subtest: project discovered and webhook registered, in 1.43 seconds)1720subtest: push delivers the webhook1721gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1722gitlab # [ 443.272102] gitaly[923]: time="2026-11-10T11:06:23.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6FJ5R4ABW5Z5JZS8SQD7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-10T11:06:53.100" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:23.800" grpc.time_ms=20.285 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1723gitlab # [ 443.314510] gitaly[923]: time="2026-11-10T11:06:23.866Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6FJ5R4ABW5Z5JZS8SQD7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:06:53.164" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:06:23.865" grpc.time_ms=0.888 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1724gitlab # [ 443.335142] gitaly[923]: time="2026-11-10T11:06:23.887Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6FJ5R4ABW5Z5JZS8SQD7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-10T11:06:33.005" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:06:23.875" grpc.time_ms=11.121 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725gitlab # [ 443.393486] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2591726gitlab # [ 443.563692] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1611727gitlab # [ 443.571217] nixbot[4291]: {"ts": "2026-11-10T11:06:24.122Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1728gitlab # [ 443.766056] gitaly[923]: time="2026-11-10T11:06:24.318Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72J6G2PBWWEKWYJE9KSCYQS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.303" grpc.time_ms=14.062 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1729gitlab # [ 443.787750] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1051730gitlab # [ 443.980206] gitaly[923]: time="2026-11-10T11:06:24.532Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6G6105Z74E8RMMJSBKVR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.518" pid=923 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1731gitlab # [ 443.997596] gitaly[923]: time="2026-11-10T11:06:24.532Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6G6105Z74E8RMMJSBKVR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.518" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 444.019619] gitlab-workhorse[829]: time="2026-11-10T11:06:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131733gitlab # [ 444.022693] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 2111734gitlab # [ 444.025626] gitaly[923]: time="2026-11-10T11:06:24.533Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6G6105Z74E8RMMJSBKVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.518" grpc.time_ms=14.416 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1735gitlab # [ 444.090780] gitaly[923]: time="2026-11-10T11:06:24.642Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GCWTVJPXBP3HMZ9JND0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.619" pid=923 remote_ip="2001:db8:1::1" request_sha=2e29ef8b4f777e5d0f572b4b64451c41f2e31a1d response_bytes=223 span.kind=server system=grpc1736gitlab # [ 444.104335] gitaly[923]: time="2026-11-10T11:06:24.643Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GCWTVJPXBP3HMZ9JND0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.619" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1737gitlab # [ 444.121532] gitlab-workhorse[829]: time="2026-11-10T11:06:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231738gitlab # [ 444.123695] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1031739gitlab # [ 444.126029] gitaly[923]: time="2026-11-10T11:06:24.643Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GCWTVJPXBP3HMZ9JND0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.619" grpc.time_ms=23.874 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1740gitlab # [ 444.232618] gitaly[923]: time="2026-11-10T11:06:24.784Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M72J6GHHYND2TDN03KMHV7JM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.771" grpc.time_ms=13.178 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1741gitlab # [ 444.255310] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 951742gitlab # [ 444.344749] gitaly[923]: time="2026-11-10T11:06:24.895Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GMKHN7SKFCJKT32580C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.883" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1743gitlab # [ 444.360615] gitaly[923]: time="2026-11-10T11:06:24.895Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GMKHN7SKFCJKT32580C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.883" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1744gitlab # [ 444.378675] gitlab-workhorse[829]: time="2026-11-10T11:06:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671745gitlab # [ 444.381171] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1091746gitlab # [ 444.383405] gitaly[923]: time="2026-11-10T11:06:24.898Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GMKHN7SKFCJKT32580C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:24.883" grpc.time_ms=12.415 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1747gitlab # [ 444.464558] gitaly[923]: time="2026-11-10T11:06:25.016Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M72J6GS9G2HHF2QA00KVC3XV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:25.004" grpc.time_ms=11.589 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1748gitlab # [ 444.487923] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 791749gitlab # [ 444.576615] gitaly[923]: time="2026-11-10T11:06:25.128Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GVVCNQH2VTJZ019Q3ZM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:25.107" pid=923 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1750gitlab # [ 444.594634] gitaly[923]: time="2026-11-10T11:06:25.128Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GVVCNQH2VTJZ019Q3ZM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:25.107" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1751gitlab # [ 444.616966] gitlab-workhorse[829]: time="2026-11-10T11:06:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091752gitlab # [ 444.620247] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1101753gitlab # [ 444.623006] gitaly[923]: time="2026-11-10T11:06:25.129Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6GVVCNQH2VTJZ019Q3ZM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:25.107" grpc.time_ms=21.9 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1754gitlab # [ 444.719941] gitaly[923]: time="2026-11-10T11:06:25.271Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:06:55.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:06:25.270" grpc.time_ms=0.93 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755gitlab # [ 444.772854] gitaly[923]: time="2026-11-10T11:06:25.324Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T11:06:55.111" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:25.311" grpc.time_ms=13.172 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756gitlab # [ 444.809189] gitaly[923]: time="2026-11-10T11:06:25.361Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:06:35.098" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:25.348" grpc.time_ms=12.124 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1757gitlab # [ 444.835859] gitaly[923]: time="2026-11-10T11:06:25.387Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:06:35.125" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:25.375" grpc.time_ms=11.92 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758gitlab # [ 444.951637] gitaly[923]: time="2026-11-10T11:06:25.503Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T11:06:35.252" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:25.503" grpc.time_ms=0.479 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759gitlab # [ 444.983239] gitaly[923]: time="2026-11-10T11:06:25.535Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T11:06:35.271" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:25.522" grpc.time_ms=12.607 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760gitlab # [ 445.039689] gitaly[923]: time="2026-11-10T11:06:25.591Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:06:25.590" grpc.time_ms=1.407 pid=923 span.kind=server system=grpc1761gitlab # [ 445.080428] gitaly[923]: time="2026-11-10T11:06:25.632Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:06:25.631" grpc.time_ms=0.764 pid=923 span.kind=server system=grpc1762gitlab # [ 445.109076] gitaly[923]: time="2026-11-10T11:06:25.661Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P diskcache=2210de95-78a4-43aa-a96b-cf9469b558b5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T11:06:35.289" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:25.540" pid=923 ref=refs/keep-around/5599f99b1c2db2684ebacd4a272932f856d9fe29 remote_ip="2001:db8:1::1" revision=5599f99b1c2db2684ebacd4a272932f856d9fe29 span.kind=server system=grpc username=root1763gitlab # [ 445.127700] gitaly[923]: time="2026-11-10T11:06:25.661Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=97 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5168 command.oublock=8 command.real_time_ms=110 command.spawn_token_fork_ms=2 command.system_time_ms=59 command.user_time_ms=38 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T11:06:35.289" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:06:25.540" grpc.time_ms=121.349 pid=923 ref=refs/keep-around/5599f99b1c2db2684ebacd4a272932f856d9fe29 remote_ip="2001:db8:1::1" revision=5599f99b1c2db2684ebacd4a272932f856d9fe29 span.kind=server system=grpc username=root1764gitlab # [ 445.149243] gitaly[923]: time="2026-11-10T11:06:25.668Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:06:55.266" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:06:25.667" grpc.time_ms=0.621 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1765gitlab # [ 445.572189] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5599f99b1c2db2684ebacd4a272932f856d9fe29 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 9471766gitlab # [ 445.588649] nixbot[4291]: {"ts": "2026-11-10T11:06:26.140Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-5599f99b1c2d-0"}1767gitlab # [ 445.878373] nixbot[4291]: {"ts": "2026-11-10T11:06:26.430Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-5599f99b1c2d-0", "argv0": "bwrap"}1768gitlab # [ 446.037478] systemd[1]: Started Nix Daemon.1769gitlab # [ 446.150584] nix-daemon[4471]: accepted connection from pid 4466, user nixbot1770gitlab # [ 450.927273] nix-daemon[4471]: accepted connection from pid 4517, user nixbot1771gitlab # [ 450.945889] nix-daemon[4471]: accepted connection from pid 4466, user nixbot1772gitlab # [ 455.151895] nix-daemon[4471]: accepted connection from pid 4466, user nixbot1773gitlab # [ 455.319432] gitaly[923]: time="2026-11-10T11:06:35.870Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6VBQ4Y3YRS6GYYK4TRF4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:07:05.468" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:06:35.869" grpc.time_ms=1.224 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774gitlab # [ 455.390936] gitaly[923]: time="2026-11-10T11:06:35.942Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M72J6VBQ4Y3YRS6GYYK4TRF4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:06:45.090" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:35.930" grpc.time_ms=12.044 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 455.501220] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5599f99b1c2db2684ebacd4a272932f856d9fe29 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2851776gitlab # [ 455.608049] gitaly[923]: time="2026-11-10T11:06:36.160Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6VN0G0K0JDBZDH88EAY5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:07:06.058" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:06:36.158" grpc.time_ms=0.954 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1777gitlab # [ 455.673563] gitaly[923]: time="2026-11-10T11:06:36.225Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6VN0G0K0JDBZDH88EAY5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:06:46.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:36.213" grpc.time_ms=12.214 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1778gitlab # [ 455.776103] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5599f99b1c2db2684ebacd4a272932f856d9fe29 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2631779gitlab # [ 455.876782] gitaly[923]: time="2026-11-10T11:06:36.428Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6VXRMRW1XFZCN50GXPH4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:07:06.027" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:06:36.427" grpc.time_ms=0.91 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 456.084343] gitaly[923]: time="2026-11-10T11:06:36.636Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M72J6VXRMRW1XFZCN50GXPH4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:06:46.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:06:36.623" grpc.time_ms=12.081 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1781gitlab # [ 456.268081] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5599f99b1c2db2684ebacd4a272932f856d9fe29 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 4751782gitlab # [ 456.562557] nixbot[4291]: {"ts": "2026-11-10T11:06:37.113Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "5599f99b1c2db2684ebacd4a272932f856d9fe29", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1783gitlab # [ 456.713168] gitaly[923]: time="2026-11-10T11:06:37.264Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=5 command.system_time_ms=0 command.user_time_ms=9 component=gitaly.StreamServerInterceptor correlation_id=01M72J6WQEJD7Q45FJEDZZ5YYW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.248" grpc.time_ms=15.446 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 456.736851] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 981785gitlab # [ 456.848155] gitaly[923]: time="2026-11-10T11:06:37.400Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6WV9YQVW8N0N6EWWZA1X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.378" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1786gitlab # [ 456.868552] gitaly[923]: time="2026-11-10T11:06:37.404Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=9 command.system_time_ms=0 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6WV9YQVW8N0N6EWWZA1X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.378" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1787gitlab # [ 456.883301] gitlab-workhorse[829]: time="2026-11-10T11:06:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671788gitlab # [ 456.885730] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1191789gitlab # [ 456.887759] gitaly[923]: time="2026-11-10T11:06:37.404Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=9 command.system_time_ms=0 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6WV9YQVW8N0N6EWWZA1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.378" grpc.time_ms=25.8 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1790gitlab # [ 456.976552] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 741791gitlab # [ 456.980230] gitaly[923]: time="2026-11-10T11:06:37.528Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M72J6X0EEBXTCMPMC3ZC5RBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.517" grpc.time_ms=10.836 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1792gitlab # [ 457.117387] gitaly[923]: time="2026-11-10T11:06:37.669Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6X2XSJMBQ67P9Y9VVF8W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.657" pid=923 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1793gitlab # [ 457.133710] gitaly[923]: time="2026-11-10T11:06:37.669Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6X2XSJMBQ67P9Y9VVF8W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.657" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1794gitlab # [ 457.153160] gitlab-workhorse[829]: time="2026-11-10T11:06:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671795gitlab # [ 457.155705] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:06:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1361796gitlab # [ 457.158480] gitaly[923]: time="2026-11-10T11:06:37.669Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6X2XSJMBQ67P9Y9VVF8W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:06:37.657" grpc.time_ms=12.016 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc1797gitlab # [ 494.281638] sidekiq-cluster[4185]: 2026-11-10T11:07:14.832Z pid=4185 tid=zch INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>4185, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425152512}1798gitlab # [ 494.712089] sidekiq-cluster[4185]: W, [2026-11-10T11:07:15.264025 #4185] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1799gitlab # [ 494.890767] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:07:15.442Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1800gitlab # [ 498.580274] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:07:19.131Z","message":"GitLab reliable fetch activated!"}1801gitlab # [ 498.580578] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:07:19.131Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1802gitlab # [ 544.205268] postgres[5246]: [5246] ERROR: relation "table_without_models" does not exist1803gitlab # [ 544.207135] postgres[5246]: [5246] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1804gitlab # [ 549.763263] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:10.314Z","message":"Booted Rails 7.2.3.1 application in production environment"}1805gitlab # [ 549.763580] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:10.314Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1806gitlab # [ 549.763657] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:10.314Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1807gitlab # [ 549.763721] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:10.314Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1808gitlab # [ 549.776111] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:10.316Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1809gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 107.27 seconds)1810gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1811gitlab # [ 550.692438] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 951812gitlab # [ 550.853078] gitaly[923]: time="2026-11-10T11:08:11.404Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RKD1VNYHWSVZ9CNRQX9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:11.391" grpc.time_ms=13.191 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1813gitlab # Enumerating objects: 5, done.1814gitlab # [ 550.867837] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 1601815gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1816gitlab # Delta compression using up to 4 threads1817gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1818gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1819gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1820gitlab # [ 551.230923] gitaly[923]: time="2026-11-10T11:08:11.782Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T11:08:41.672" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T11:08:11.772" grpc.time_ms=9.897 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 551.273158] gitaly[923]: time="2026-11-10T11:08:11.825Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W duration_ms=149ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/allowed"1822gitlab # [ 551.346232] gitaly[923]: time="2026-11-10T11:08:11.898Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W duration_ms=71ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/pre_receive"1823gitlab # [ 551.351888] gitaly[923]: time="2026-11-10T11:08:11.903Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5315 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:11.675" grpc.time_ms=228.563 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1824gitlab # [ 551.409528] gitaly[923]: time="2026-11-10T11:08:11.961Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:11.959" grpc.time_ms=1.421 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1825gitlab # [ 551.484228] gitaly[923]: time="2026-11-10T11:08:12.035Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:12.034" grpc.time_ms=1.013 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1826gitlab # [ 551.542032] gitaly[923]: time="2026-11-10T11:08:12.093Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:12.092" grpc.time_ms=1.369 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1827gitlab # [ 551.780771] gitaly[923]: time="2026-11-10T11:08:12.332Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W duration_ms=167ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/post_receive"1828gitlab # [ 551.783977] gitaly[923]: time="2026-11-10T11:08:12.333Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4809 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:12.163" grpc.time_ms=169.448 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1829gitlab # [ 551.796379] gitaly[923]: time="2026-11-10T11:08:12.346Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W diskcache=07d95d06-78b9-4826-bd59-61fdf19cd4d4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:11.602" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1830gitlab # To http://gitlab/root/test-flake.git1831gitlab # [ 551.807307] gitlab-workhorse[829]: time="2026-11-10T11:08:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521832gitlab # 5599f99..92d9e38 [ 551.809596] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:12 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 9101833gitlab # master -> master[ 551.811820] gitaly[923]: time="2026-11-10T11:08:12.346Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=322 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13736 command.oublock=32 command.real_time_ms=734 command.spawn_token_fork_ms=0 command.system_time_ms=180 command.user_time_ms=142 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:11.602" grpc.time_ms=743.88 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1834gitlab # 1835gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.30 seconds)1836gitlab: must succeed: git -C /tmp/test-flake rev-parse master1837gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1838gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1839gitlab # [ 551.961700] nixbot[4291]: {"ts": "2026-11-10T11:08:12.512Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57110 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1840gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.12 seconds)1841gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1842gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.04 seconds)1843gitlab # [ 553.004332] nixbot[4291]: {"ts": "2026-11-10T11:08:13.555Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57116 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1844gitlab # [ 553.119966] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"54e1e152ce44f7815eb7e2ed","created_at":"2026-11-10T11:06:05.297Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"608094429ab99bdb72fe1ceb07fd70de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.314Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-54e1e152ce44f7815eb7e2ed: start","job_status":"start","queue_duration_s":128.356665,"scheduling_latency_s":128.356741}1845gitlab # [ 553.202982] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.754Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1846gitlab # [ 553.221254] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"54e1e152ce44f7815eb7e2ed","created_at":"2026-11-10T11:06:05.297Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"608094429ab99bdb72fe1ceb07fd70de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.314Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-54e1e152ce44f7815eb7e2ed: done: 0.102146 sec","job_status":"done","queue_duration_s":128.356665,"scheduling_latency_s":128.356741,"redis_calls":20,"redis_duration_s":0.00634,"redis_read_bytes":907,"redis_write_bytes":2326,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003894,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1505,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001877,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.102146,"completed_at":"2026-11-10T11:08:13.772Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1847gitlab # [ 553.283298] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.775Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2fb80a4fa1ecbb6c49be0445","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2fb80a4fa1ecbb6c49be0445: start","job_status":"start","queue_duration_s":128.42037,"scheduling_latency_s":128.420404}1848gitlab # [ 553.296772] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.821Z","message":"No existing merge request to be cleaned up."}1849gitlab # [ 553.298552] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.849Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2fb80a4fa1ecbb6c49be0445","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2fb80a4fa1ecbb6c49be0445: done: 0.074431 sec","job_status":"done","queue_duration_s":128.42037,"scheduling_latency_s":128.420404,"redis_calls":14,"redis_duration_s":0.004386,"redis_read_bytes":12,"redis_write_bytes":1678,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001212,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001786,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":5,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.074431,"completed_at":"2026-11-10T11:08:13.849Z","load_balancing_strategy":"primary","db_duration_s":0.012843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1850gitlab # [ 553.341535] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.852Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e47f401a5fee90ba69b27ec5","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e47f401a5fee90ba69b27ec5: start","job_status":"start","queue_duration_s":128.496951,"scheduling_latency_s":128.49699}1851gitlab # [ 553.354584] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.870Z","message":"No existing merge request to be cleaned up."}1852gitlab # [ 553.356384] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.885Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e47f401a5fee90ba69b27ec5","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e47f401a5fee90ba69b27ec5: done: 0.033843 sec","job_status":"done","queue_duration_s":128.496951,"scheduling_latency_s":128.49699,"redis_calls":10,"redis_duration_s":0.003312,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001421,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033843,"completed_at":"2026-11-10T11:08:13.885Z","load_balancing_strategy":"primary","db_duration_s":0.003087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1853gitlab # [ 553.397132] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.887Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a22c74fc239b1b0613958d2b","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a22c74fc239b1b0613958d2b: start","job_status":"start","queue_duration_s":128.532667,"scheduling_latency_s":128.532697}1854gitlab # [ 553.410208] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.904Z","message":"No existing merge request to be cleaned up."}1855gitlab # [ 553.411899] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.918Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a22c74fc239b1b0613958d2b","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a22c74fc239b1b0613958d2b: done: 0.0303 sec","job_status":"done","queue_duration_s":128.532667,"scheduling_latency_s":128.532697,"redis_calls":10,"redis_duration_s":0.002746,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001161,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0303,"completed_at":"2026-11-10T11:08:13.918Z","load_balancing_strategy":"primary","db_duration_s":0.003144,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1856gitlab # [ 553.453755] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.920Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0843689107a3938be4f05130","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0843689107a3938be4f05130: start","job_status":"start","queue_duration_s":128.565166,"scheduling_latency_s":128.565199}1857gitlab # [ 553.466658] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.938Z","message":"No existing merge request to be cleaned up."}1858gitlab # [ 553.468399] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.948Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0843689107a3938be4f05130","created_at":"2026-11-10T11:06:05.289Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-6b867ec6b4fc44a5","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c51179faf0b9c56ddf5ce1962a938877","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.355Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0843689107a3938be4f05130: done: 0.028598 sec","job_status":"done","queue_duration_s":128.565166,"scheduling_latency_s":128.565199,"redis_calls":10,"redis_duration_s":0.002672,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001524,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028598,"completed_at":"2026-11-10T11:08:13.948Z","load_balancing_strategy":"primary","db_duration_s":0.002685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859gitlab # [ 553.508139] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2040407b6e165daf7db057ed","created_at":"2026-11-10T11:06:05.356Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7baffd0458ef05ab23679c66874a71e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.367Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2040407b6e165daf7db057ed: start","job_status":"start","queue_duration_s":128.582794,"scheduling_latency_s":128.582821}1860gitlab # [ 553.523293] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2040407b6e165daf7db057ed","created_at":"2026-11-10T11:06:05.356Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7baffd0458ef05ab23679c66874a71e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.367Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2040407b6e165daf7db057ed: done: 0.030495 sec","job_status":"done","queue_duration_s":128.582794,"scheduling_latency_s":128.582821,"redis_calls":13,"redis_duration_s":0.003333,"redis_read_bytes":11,"redis_write_bytes":1783,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001416,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1007,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001462,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030495,"completed_at":"2026-11-10T11:08:13.981Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861gitlab # [ 553.569697] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:13.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2fda237b6e7ba9d38628a8b6","created_at":"2026-11-10T11:06:05.410Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a514259ce8bc77703ce24c37da3d0e5","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.421Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2fda237b6e7ba9d38628a8b6: start","job_status":"start","queue_duration_s":128.561493,"scheduling_latency_s":128.561522}1862gitlab # [ 553.585305] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2fda237b6e7ba9d38628a8b6","created_at":"2026-11-10T11:06:05.410Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a514259ce8bc77703ce24c37da3d0e5","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.421Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2fda237b6e7ba9d38628a8b6: done: 0.04885 sec","job_status":"done","queue_duration_s":128.561493,"scheduling_latency_s":128.561522,"redis_calls":14,"redis_duration_s":0.004324,"redis_read_bytes":12,"redis_write_bytes":1967,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001248,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001816,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00126,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01398956,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.04885,"completed_at":"2026-11-10T11:08:14.031Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005775,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1863gitlab # [ 553.632227] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b42904214d9e895e9777f62c","created_at":"2026-11-10T11:06:05.470Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2c1a4fb538236236f0058860ef2b45c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.484Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b42904214d9e895e9777f62c: start","job_status":"start","queue_duration_s":128.549463,"scheduling_latency_s":128.549497}1864gitlab # [ 553.647527] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b42904214d9e895e9777f62c","created_at":"2026-11-10T11:06:05.470Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2c1a4fb538236236f0058860ef2b45c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.484Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b42904214d9e895e9777f62c: done: 0.10131 sec","job_status":"done","queue_duration_s":128.549463,"scheduling_latency_s":128.549497,"redis_calls":13,"redis_duration_s":0.005376,"redis_read_bytes":11,"redis_write_bytes":1921,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001462,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":752,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00208,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00144582,"exclusive_lock_hold_duration_s":0.04596304,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.10131,"completed_at":"2026-11-10T11:08:14.135Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865gitlab # [ 553.695789] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8c23a23a63cf35024a6e6d3f","created_at":"2026-11-10T11:06:05.513Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a73cdc57ba9ddeb3451929e4559feac8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.520Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8c23a23a63cf35024a6e6d3f: start","job_status":"start","queue_duration_s":128.617481,"scheduling_latency_s":128.617513}1866gitlab # [ 553.710457] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8c23a23a63cf35024a6e6d3f","created_at":"2026-11-10T11:06:05.513Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a73cdc57ba9ddeb3451929e4559feac8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.520Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8c23a23a63cf35024a6e6d3f: done: 0.045063 sec","job_status":"done","queue_duration_s":128.617481,"scheduling_latency_s":128.617513,"redis_calls":13,"redis_duration_s":0.004607,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001225,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":708,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007927,"exclusive_lock_hold_duration_s":0.01102064,"duration_s":0.045063,"completed_at":"2026-11-10T11:08:14.182Z","load_balancing_strategy":"primary","db_duration_s":0.005784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867gitlab # [ 553.754592] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"65f04c6ea00b2a240ad58450","created_at":"2026-11-10T11:06:05.557Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af0a470c9836cb580714ea61d88fb879","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.567Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-65f04c6ea00b2a240ad58450: start","job_status":"start","queue_duration_s":128.618754,"scheduling_latency_s":128.618783}1868gitlab # [ 553.769787] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"65f04c6ea00b2a240ad58450","created_at":"2026-11-10T11:06:05.557Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af0a470c9836cb580714ea61d88fb879","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.567Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-65f04c6ea00b2a240ad58450: done: 0.051989 sec","job_status":"done","queue_duration_s":128.618754,"scheduling_latency_s":128.618783,"redis_calls":13,"redis_duration_s":0.005476,"redis_read_bytes":11,"redis_write_bytes":1803,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001454,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":716,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002236,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001786,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00112394,"exclusive_lock_hold_duration_s":0.00735856,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.051989,"completed_at":"2026-11-10T11:08:14.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1869gitlab # [ 553.815113] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"45f98a609097a7f9808d1e88","created_at":"2026-11-10T11:06:05.579Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b033938638d5e689a36c3fa9c3062bf3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:06:05.589Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-45f98a609097a7f9808d1e88: start","job_status":"start","queue_duration_s":128.651343,"scheduling_latency_s":128.651384}1870gitlab # [ 553.829502] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.274Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1871gitlab # [ 553.833011] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"45f98a609097a7f9808d1e88","created_at":"2026-11-10T11:06:05.579Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b033938638d5e689a36c3fa9c3062bf3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:06:05.589Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-45f98a609097a7f9808d1e88: done: 0.048903 sec","job_status":"done","queue_duration_s":128.651343,"scheduling_latency_s":128.651384,"redis_calls":11,"redis_duration_s":0.004535,"redis_read_bytes":9,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001567,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00077,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048903,"completed_at":"2026-11-10T11:08:14.289Z","load_balancing_strategy":"primary","db_duration_s":0.001912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1872gitlab # [ 553.877412] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"889e4fa624cc8638a6459e0f","created_at":"2026-11-10T11:06:05.613Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2dc6dfe2bf2c93f77662ecd22bd0121","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.624Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-889e4fa624cc8638a6459e0f: start","job_status":"start","queue_duration_s":128.668295,"scheduling_latency_s":128.668335}1873gitlab # [ 553.892770] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"889e4fa624cc8638a6459e0f","created_at":"2026-11-10T11:06:05.613Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2dc6dfe2bf2c93f77662ecd22bd0121","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.624Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-889e4fa624cc8638a6459e0f: done: 0.052445 sec","job_status":"done","queue_duration_s":128.668295,"scheduling_latency_s":128.668335,"redis_calls":11,"redis_duration_s":0.0035269999999999998,"redis_read_bytes":9,"redis_write_bytes":1522,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001315,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":736,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001602,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00061,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052445,"completed_at":"2026-11-10T11:08:14.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1874gitlab # [ 553.937059] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5bf325e1f0349024da4e5ba0","created_at":"2026-11-10T11:06:05.630Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eea3ddf0f26e211f5f5d8c2f310c5453","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:06:05.644Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5bf325e1f0349024da4e5ba0: start","job_status":"start","queue_duration_s":128.702901,"scheduling_latency_s":128.702936}1875gitlab # [ 553.951174] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.374Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1876gitlab # [ 553.955237] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"5bf325e1f0349024da4e5ba0","created_at":"2026-11-10T11:06:05.630Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eea3ddf0f26e211f5f5d8c2f310c5453","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:06:05.644Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-5bf325e1f0349024da4e5ba0: done: 0.043491 sec","job_status":"done","queue_duration_s":128.702901,"scheduling_latency_s":128.702936,"redis_calls":11,"redis_duration_s":0.005337,"redis_read_bytes":9,"redis_write_bytes":1557,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001727,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":756,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043491,"completed_at":"2026-11-10T11:08:14.390Z","load_balancing_strategy":"primary","db_duration_s":0.001754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1877gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1878gitlab # [ 553.998484] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3643cce541c123df052d6fb6","created_at":"2026-11-10T11:06:05.668Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa6a24768c4052334a57bc6ba2d129d8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.675Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3643cce541c123df052d6fb6: start","job_status":"start","queue_duration_s":128.71851,"scheduling_latency_s":128.718555}1879gitlab # [ 554.013617] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.423Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1880gitlab # [ 554.016909] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.438Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3643cce541c123df052d6fb6","created_at":"2026-11-10T11:06:05.668Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa6a24768c4052334a57bc6ba2d129d8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.675Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3643cce541c123df052d6fb6: done: 0.044543 sec","job_status":"done","queue_duration_s":128.71851,"scheduling_latency_s":128.718555,"redis_calls":11,"redis_duration_s":0.003474,"redis_read_bytes":9,"redis_write_bytes":1598,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00135,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":772,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001572,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044543,"completed_at":"2026-11-10T11:08:14.438Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1881gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.06 seconds)1882gitlab # [ 554.060816] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.440Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7f762d0afd6da8bb2fe3721d","created_at":"2026-11-10T11:06:05.697Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc5175e26d19556af57b0f309785796","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:06:05.709Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7f762d0afd6da8bb2fe3721d: start","job_status":"start","queue_duration_s":128.731079,"scheduling_latency_s":128.731121}1883gitlab # [ 554.075586] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7f762d0afd6da8bb2fe3721d","created_at":"2026-11-10T11:06:05.697Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc5175e26d19556af57b0f309785796","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:06:05.709Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7f762d0afd6da8bb2fe3721d: done: 0.09387 sec","job_status":"done","queue_duration_s":128.731079,"scheduling_latency_s":128.731121,"redis_calls":11,"redis_duration_s":0.003909,"redis_read_bytes":9,"redis_write_bytes":1449,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001167,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":708,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001997,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000745,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.09387,"completed_at":"2026-11-10T11:08:14.534Z","load_balancing_strategy":"primary","db_duration_s":0.01835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1884gitlab # [ 554.118905] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d062b3e7fa0496e989f5e61d","created_at":"2026-11-10T11:06:05.739Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44d809cea997f527a9f2b171f0107d0b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.754Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d062b3e7fa0496e989f5e61d: start","job_status":"start","queue_duration_s":128.782056,"scheduling_latency_s":128.782095}1885gitlab # [ 554.134896] nixbot[4291]: {"ts": "2026-11-10T11:08:14.615Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57128 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1886gitlab # [ 554.138346] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d062b3e7fa0496e989f5e61d","created_at":"2026-11-10T11:06:05.739Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44d809cea997f527a9f2b171f0107d0b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.754Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d062b3e7fa0496e989f5e61d: done: 0.03894 sec","job_status":"done","queue_duration_s":128.782056,"scheduling_latency_s":128.782095,"redis_calls":11,"redis_duration_s":0.003359,"redis_read_bytes":9,"redis_write_bytes":1607,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001212,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000565,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03894,"completed_at":"2026-11-10T11:08:14.575Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1887gitlab # [ 554.183326] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e062dd72df83be923669e1f8","created_at":"2026-11-10T11:06:05.795Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36411fa9379b569766229bc6cb725f82","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.808Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e062dd72df83be923669e1f8: start","job_status":"start","queue_duration_s":128.773269,"scheduling_latency_s":128.773314}1888gitlab # [ 554.199048] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e062dd72df83be923669e1f8","created_at":"2026-11-10T11:06:05.795Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36411fa9379b569766229bc6cb725f82","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.808Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e062dd72df83be923669e1f8: done: 0.051704 sec","job_status":"done","queue_duration_s":128.773269,"scheduling_latency_s":128.773314,"redis_calls":13,"redis_duration_s":0.005197,"redis_read_bytes":11,"redis_write_bytes":1855,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002121,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":728,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001565,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001511,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00094882,"exclusive_lock_hold_duration_s":0.00967828,"duration_s":0.051704,"completed_at":"2026-11-10T11:08:14.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1889gitlab # [ 554.244168] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:14.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0be33d201e02aa689170e531","created_at":"2026-11-10T11:06:05.837Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"724feef485b9dd9930dd962380afa5b5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.847Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0be33d201e02aa689170e531: start","job_status":"start","queue_duration_s":128.789214,"scheduling_latency_s":128.789252}1890gitlab # [ 554.514680] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0be33d201e02aa689170e531","created_at":"2026-11-10T11:06:05.837Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"724feef485b9dd9930dd962380afa5b5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.847Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0be33d201e02aa689170e531: done: 0.429413 sec","job_status":"done","queue_duration_s":128.789214,"scheduling_latency_s":128.789252,"redis_calls":566,"redis_duration_s":0.097989,"redis_read_bytes":564,"redis_write_bytes":49420,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001192,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":660,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001446,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.095351,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.377005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.429413,"completed_at":"2026-11-10T11:08:15.065Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1891gitlab # [ 554.579610] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"06360a346c238b89eb2662fb","created_at":"2026-11-10T11:06:05.895Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1487decfe57f902d81d850fa1ad7692","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.904Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-06360a346c238b89eb2662fb: start","job_status":"start","queue_duration_s":129.164773,"scheduling_latency_s":129.164816}1892gitlab # [ 554.596519] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"06360a346c238b89eb2662fb","created_at":"2026-11-10T11:06:05.895Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1487decfe57f902d81d850fa1ad7692","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.904Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-06360a346c238b89eb2662fb: done: 0.05494 sec","job_status":"done","queue_duration_s":129.164773,"scheduling_latency_s":129.164816,"redis_calls":11,"redis_duration_s":0.0036090000000000002,"redis_read_bytes":9,"redis_write_bytes":1341,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001332,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":660,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05494,"completed_at":"2026-11-10T11:08:15.123Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1893gitlab # [ 554.641204] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.127Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0f226acded51babf07993854","created_at":"2026-11-10T11:06:05.935Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"804a585782c35d037788a83e7c4b4637","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.958Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0f226acded51babf07993854: start","job_status":"start","queue_duration_s":129.168216,"scheduling_latency_s":129.168259}1894gitlab # [ 554.654817] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.167Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0f226acded51babf07993854","created_at":"2026-11-10T11:06:05.935Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"804a585782c35d037788a83e7c4b4637","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:05.958Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0f226acded51babf07993854: done: 0.040326 sec","job_status":"done","queue_duration_s":129.168216,"scheduling_latency_s":129.168259,"redis_calls":12,"redis_duration_s":0.00403,"redis_read_bytes":136,"redis_write_bytes":1205,"redis_cache_calls":2,"redis_cache_duration_s":0.00074,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001046,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":416,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040326,"completed_at":"2026-11-10T11:08:15.167Z","load_balancing_strategy":"primary","db_duration_s":0.000949,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1895gitlab # [ 554.700543] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5f3eb1f66e2617c483686ea6","created_at":"2026-11-10T11:06:05.991Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fd0dc9d641fb90cc9fdbf6e3e4f867a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.006Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5f3eb1f66e2617c483686ea6: start","job_status":"start","queue_duration_s":129.1637,"scheduling_latency_s":129.163739}1896gitlab # [ 554.715816] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.195Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1897gitlab # [ 554.719781] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5f3eb1f66e2617c483686ea6","created_at":"2026-11-10T11:06:05.991Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fd0dc9d641fb90cc9fdbf6e3e4f867a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.006Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5f3eb1f66e2617c483686ea6: done: 0.037778 sec","job_status":"done","queue_duration_s":129.1637,"scheduling_latency_s":129.163739,"redis_calls":11,"redis_duration_s":0.003125,"redis_read_bytes":9,"redis_write_bytes":1498,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001176,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":732,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037778,"completed_at":"2026-11-10T11:08:15.207Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1898gitlab # [ 554.764167] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e12b04e0cd7a3408a650f8ea","created_at":"2026-11-10T11:06:06.035Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d61680b2d0d2adf34460a07056bda7c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.044Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e12b04e0cd7a3408a650f8ea: start","job_status":"start","queue_duration_s":129.165454,"scheduling_latency_s":129.165492}1899gitlab # [ 554.779349] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e12b04e0cd7a3408a650f8ea","created_at":"2026-11-10T11:06:06.035Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d61680b2d0d2adf34460a07056bda7c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.044Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e12b04e0cd7a3408a650f8ea: done: 0.100338 sec","job_status":"done","queue_duration_s":129.165454,"scheduling_latency_s":129.165492,"redis_calls":15,"redis_duration_s":0.00516,"redis_read_bytes":13,"redis_write_bytes":2181,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002114,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1019,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001577,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":9,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00096576,"exclusive_lock_hold_duration_s":0.05451596,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.100338,"completed_at":"2026-11-10T11:08:15.309Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037674,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1900gitlab # [ 554.826285] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.312Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3053adf0e875b8d7c2519356","created_at":"2026-11-10T11:06:06.072Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5550d5a41cb3195521b026965d9da68f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.079Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3053adf0e875b8d7c2519356: start","job_status":"start","queue_duration_s":129.233145,"scheduling_latency_s":129.233184}1901gitlab # [ 554.841607] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.338Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1902gitlab # [ 554.844917] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3053adf0e875b8d7c2519356","created_at":"2026-11-10T11:06:06.072Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5550d5a41cb3195521b026965d9da68f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.079Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3053adf0e875b8d7c2519356: done: 0.038875 sec","job_status":"done","queue_duration_s":129.233145,"scheduling_latency_s":129.233184,"redis_calls":11,"redis_duration_s":0.003436,"redis_read_bytes":9,"redis_write_bytes":1507,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001354,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":736,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038875,"completed_at":"2026-11-10T11:08:15.351Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1903gitlab # [ 554.889956] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.353Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"6b24ae154741087f558624cc","created_at":"2026-11-10T11:06:06.108Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec2c1ae7a92c837f92c58c4330c3bf90","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.134Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-6b24ae154741087f558624cc: start","job_status":"start","queue_duration_s":129.219497,"scheduling_latency_s":129.219531}1904gitlab # [ 554.905065] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.385Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"6b24ae154741087f558624cc","created_at":"2026-11-10T11:06:06.108Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec2c1ae7a92c837f92c58c4330c3bf90","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.134Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-6b24ae154741087f558624cc: done: 0.032116 sec","job_status":"done","queue_duration_s":129.219497,"scheduling_latency_s":129.219531,"redis_calls":13,"redis_duration_s":0.003278,"redis_read_bytes":139,"redis_write_bytes":1372,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000863,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":458,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001428,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041958,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.032116,"completed_at":"2026-11-10T11:08:15.385Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000861,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1905gitlab # [ 554.951953] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3907bd9550189a5075dac0f5","created_at":"2026-11-10T11:06:06.171Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.183Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3907bd9550189a5075dac0f5: start","job_status":"start","queue_duration_s":129.205038,"scheduling_latency_s":129.205068}1906gitlab # [ 554.997887] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3907bd9550189a5075dac0f5","created_at":"2026-11-10T11:06:06.171Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.183Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3907bd9550189a5075dac0f5: done: 0.161346 sec","job_status":"done","queue_duration_s":129.205038,"scheduling_latency_s":129.205068,"redis_calls":72,"redis_duration_s":0.018322,"redis_read_bytes":120,"redis_write_bytes":42965,"redis_feature_flag_calls":12,"redis_feature_flag_duration_s":0.003197,"redis_feature_flag_read_bytes":12,"redis_feature_flag_write_bytes":1856,"redis_queues_calls":2,"redis_queues_duration_s":0.000736,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013304,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.141656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.161346,"completed_at":"2026-11-10T11:08:15.549Z","load_balancing_strategy":"primary","db_duration_s":0.003245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1907gitlab # [ 555.042548] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.551Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0930c97e1917ad61fca05999","created_at":"2026-11-10T11:06:06.207Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3223d5992c6583e4a19b3ebca0044cc9","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.215Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0930c97e1917ad61fca05999: start","job_status":"start","queue_duration_s":129.335951,"scheduling_latency_s":129.335984}1908gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1909gitlab # [ 555.057445] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.604Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0930c97e1917ad61fca05999","created_at":"2026-11-10T11:06:06.207Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3223d5992c6583e4a19b3ebca0044cc9","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.215Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0930c97e1917ad61fca05999: done: 0.05329 sec","job_status":"done","queue_duration_s":129.335951,"scheduling_latency_s":129.335984,"redis_calls":17,"redis_duration_s":0.005213,"redis_read_bytes":15,"redis_write_bytes":2228,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003493,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1517,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000475,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05329,"completed_at":"2026-11-10T11:08:15.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1910gitlab # [ 555.101488] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8a6e1b9a179c14c19f2ad7b7","created_at":"2026-11-10T11:06:06.230Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa166b5153dc69e0ea28f350da517589","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.250Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8a6e1b9a179c14c19f2ad7b7: start","job_status":"start","queue_duration_s":129.356114,"scheduling_latency_s":129.356146}1911gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.05 seconds)1912gitlab # [ 555.116178] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8a6e1b9a179c14c19f2ad7b7","created_at":"2026-11-10T11:06:06.230Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-58c807c2d3b84b6f","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa166b5153dc69e0ea28f350da517589","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:06.250Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8a6e1b9a179c14c19f2ad7b7: done: 0.050156 sec","job_status":"done","queue_duration_s":129.356114,"scheduling_latency_s":129.356146,"redis_calls":11,"redis_duration_s":0.002918,"redis_read_bytes":9,"redis_write_bytes":1594,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000978,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":768,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.050156,"completed_at":"2026-11-10T11:08:15.656Z","load_balancing_strategy":"primary","db_duration_s":0.013381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1913gitlab # [ 555.161628] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.658Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a3fc229fd1acd85eb969e823","created_at":"2026-11-10T11:06:09.669Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:09.728Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a3fc229fd1acd85eb969e823: start","job_status":"start","queue_duration_s":125.930543,"scheduling_latency_s":125.930576}1914gitlab # [ 555.177471] nixbot[4291]: {"ts": "2026-11-10T11:08:15.666Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57144 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1915gitlab # [ 555.267828] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.819Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a3fc229fd1acd85eb969e823","created_at":"2026-11-10T11:06:09.669Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:09.728Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a3fc229fd1acd85eb969e823: done: 0.160614 sec","job_status":"done","queue_duration_s":125.930543,"scheduling_latency_s":125.930576,"redis_calls":13,"redis_duration_s":0.004458,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000993,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001934,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001531,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11056358,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.160614,"completed_at":"2026-11-10T11:08:15.819Z","load_balancing_strategy":"primary","db_duration_s":0.04081,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1916gitlab # [ 555.338527] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.821Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"dd1ee461fb7bcdc16d033ce5","created_at":"2026-11-10T11:06:09.753Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:09.766Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-dd1ee461fb7bcdc16d033ce5: start","job_status":"start","queue_duration_s":126.054895,"scheduling_latency_s":126.054928}1917gitlab # [ 555.414494] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.965Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"dd1ee461fb7bcdc16d033ce5","created_at":"2026-11-10T11:06:09.753Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:09.766Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-dd1ee461fb7bcdc16d033ce5: done: 0.1441 sec","job_status":"done","queue_duration_s":126.054895,"scheduling_latency_s":126.054928,"redis_calls":14,"redis_duration_s":0.004318,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000874,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001874,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00157,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10663705999999999,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.1441,"completed_at":"2026-11-10T11:08:15.965Z","load_balancing_strategy":"primary","db_duration_s":0.029088,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1918gitlab # [ 555.479369] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:15.968Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ef79e0a34b5ffa3e1ecf8f30","created_at":"2026-11-10T11:06:10.489Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:10.499Z","job_size_bytes":5,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ef79e0a34b5ffa3e1ecf8f30: start","job_status":"start","queue_duration_s":125.468285,"scheduling_latency_s":125.468316}1919gitlab # [ 555.641938] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:16.193Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ef79e0a34b5ffa3e1ecf8f30","created_at":"2026-11-10T11:06:10.489Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:10.499Z","job_size_bytes":5,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ef79e0a34b5ffa3e1ecf8f30: done: 0.225072 sec","job_status":"done","queue_duration_s":125.468285,"scheduling_latency_s":125.468316,"redis_calls":12,"redis_duration_s":0.003821,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000852,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002464,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.182531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00085974,"duration_s":0.225072,"completed_at":"2026-11-10T11:08:16.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035471,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1920gitlab # [ 555.715105] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:16.195Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"dd1b16adecdd5e79c4e0855e","created_at":"2026-11-10T11:06:10.542Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:10.550Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-dd1b16adecdd5e79c4e0855e: start","job_status":"start","queue_duration_s":125.6448,"scheduling_latency_s":125.644832}1921gitlab # [ 555.966769] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:16.518Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"dd1b16adecdd5e79c4e0855e","created_at":"2026-11-10T11:06:10.542Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:10.550Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-dd1b16adecdd5e79c4e0855e: done: 0.322841 sec","job_status":"done","queue_duration_s":125.6448,"scheduling_latency_s":125.644832,"redis_calls":11,"redis_duration_s":0.003308,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000833,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.267,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.267,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.235055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005403,"duration_s":0.322841,"completed_at":"2026-11-10T11:08:16.518Z","load_balancing_strategy":"primary","db_duration_s":0.0943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1922gitlab # [ 556.035862] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:16.520Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ebb7d3315053889666d53f09","created_at":"2026-11-10T11:06:10.554Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:10.562Z","job_size_bytes":89,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ebb7d3315053889666d53f09: start","job_status":"start","queue_duration_s":125.958044,"scheduling_latency_s":125.958075}1923gitlab # [ 556.056880] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:16.577Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ebb7d3315053889666d53f09","created_at":"2026-11-10T11:06:10.554Z","trace_propagation_headers":{"sentry-trace":"29ab8b8cc074424d9b87ba0eb85b7a46-01a1295009be4236","baggage":"sentry-trace_id=29ab8b8cc074424d9b87ba0eb85b7a46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J60JXBF04HJ5XKB8Z8RQ6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:10.562Z","job_size_bytes":89,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ebb7d3315053889666d53f09: done: 0.056382 sec","job_status":"done","queue_duration_s":125.958044,"scheduling_latency_s":125.958075,"redis_calls":11,"redis_duration_s":0.002947,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000881,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038162,"duration_s":0.056382,"completed_at":"2026-11-10T11:08:16.576Z","load_balancing_strategy":"primary","db_duration_s":0.008471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1924gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1925gitlab # [ 556.104301] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:16.579Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5599f99b1c2db2684ebacd4a272932f856d9fe29 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"06af88d3393d0b94203ad29a","created_at":"2026-11-10T11:06:18.798Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-4b037696acca466c","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:18.810Z","job_size_bytes":185,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-06af88d3393d0b94203ad29a: start","job_status":"start","queue_duration_s":117.768859,"scheduling_latency_s":117.768888}1926gitlab # [ 556.122014] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:16.605Z","message":"changes: \"0000000000000000000000000000000000000000 5599f99b1c2db2684ebacd4a272932f856d9fe29 refs/heads/master\\n\""}1927gitlab # [ 556.125975] gitaly[923]: time="2026-11-10T11:08:16.659Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:26.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:16.645" grpc.time_ms=13.073 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab # [ 556.139796] gitaly[923]: time="2026-11-10T11:08:16.679Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:08:26.027" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:16.667" grpc.time_ms=11.824 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 556.165640] nixbot[4291]: {"ts": "2026-11-10T11:08:16.717Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57156 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1930gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.05 seconds)1931gitlab # [ 556.205372] gitaly[923]: time="2026-11-10T11:08:16.757Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T11:08:26.105" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:16.745" grpc.time_ms=11.871 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932gitlab # [ 556.261318] gitaly[923]: time="2026-11-10T11:08:16.813Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3936 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:16.812" grpc.time_ms=0.677 pid=923 span.kind=server system=grpc1933gitlab # [ 556.292241] gitaly[923]: time="2026-11-10T11:08:16.843Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:16.843" grpc.time_ms=0.602 pid=923 span.kind=server system=grpc1934gitlab # [ 556.312065] gitaly[923]: time="2026-11-10T11:08:16.864Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ diskcache=cd5e2b99-8dd5-49dc-8755-6d8a9a775429 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T11:08:26.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:16.778" pid=923 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1935gitlab # [ 556.330165] gitaly[923]: time="2026-11-10T11:08:16.864Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4916 command.oublock=8 command.real_time_ms=79 command.spawn_token_fork_ms=0 command.system_time_ms=49 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T11:08:26.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:16.778" grpc.time_ms=86.204 pid=923 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1936gitlab # [ 556.352342] gitaly[923]: time="2026-11-10T11:08:16.888Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T11:08:26.247" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:16.888" grpc.time_ms=0.342 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937gitlab # [ 556.367342] gitaly[923]: time="2026-11-10T11:08:16.912Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=860 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T11:08:26.255" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:16.895" grpc.time_ms=16.508 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938gitlab # [ 556.707154] gitaly[923]: time="2026-11-10T11:08:17.259Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-10T11:08:47.050" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:17.250" grpc.time_ms=8.496 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1939gitlab # [ 556.760908] gitaly[923]: time="2026-11-10T11:08:17.312Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T11:08:27.001" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:17.312" grpc.time_ms=0.463 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1940gitlab # [ 556.774035] gitaly[923]: time="2026-11-10T11:08:17.325Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T11:08:27.007" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T11:08:17.317" grpc.time_ms=8.157 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1941gitlab # [ 556.814419] gitaly[923]: time="2026-11-10T11:08:17.366Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:27.043" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:17.353" grpc.time_ms=12.48 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1942gitlab # [ 556.837969] gitaly[923]: time="2026-11-10T11:08:17.389Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:27.066" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:17.377" grpc.time_ms=12.594 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1943gitlab # [ 556.884190] gitaly[923]: time="2026-11-10T11:08:17.436Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T11:08:47.214" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:17.414" grpc.time_ms=21.18 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1944gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1945gitlab # [ 557.212338] nixbot[4291]: {"ts": "2026-11-10T11:08:17.763Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57158 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1946gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.05 seconds)1947gitlab # [ 557.627728] gitaly[923]: time="2026-11-10T11:08:18.179Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T11:08:28.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:18.177" grpc.time_ms=1.866 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1948gitlab # [ 557.689258] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:18.239Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5599f99b1c2db2684ebacd4a272932f856d9fe29 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"06af88d3393d0b94203ad29a","created_at":"2026-11-10T11:06:18.798Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-4b037696acca466c","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:06:18.810Z","job_size_bytes":185,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-06af88d3393d0b94203ad29a: done: 1.660567 sec","job_status":"done","queue_duration_s":117.768859,"scheduling_latency_s":117.768888,"gitaly_calls":13,"gitaly_duration_s":0.262376,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.069998,"redis_read_bytes":2108,"redis_write_bytes":21751,"redis_cache_calls":15,"redis_cache_duration_s":0.006642,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.009715,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002667,"redis_queues_read_bytes":15,"redis_queues_write_bytes":7839,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005259,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.025102,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.020613,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.369,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.413,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.084,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.268678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00130506,"exclusive_lock_hold_duration_s":0.038632679999999996,"duration_s":1.660567,"completed_at":"2026-11-10T11:08:18.239Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.14182,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1949gitlab # [ 557.757366] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:18.242Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f3aa5509dd73489190c8ca13","created_at":"2026-11-10T11:06:25.674Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-e15d7fc8a5334cc4","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:25.685Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f3aa5509dd73489190c8ca13: start","job_status":"start","queue_duration_s":112.556725,"scheduling_latency_s":112.556769}1950gitlab # [ 557.837091] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:18.388Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f3aa5509dd73489190c8ca13","created_at":"2026-11-10T11:06:25.674Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-e15d7fc8a5334cc4","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:25.685Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f3aa5509dd73489190c8ca13: done: 0.145939 sec","job_status":"done","queue_duration_s":112.556725,"scheduling_latency_s":112.556769,"redis_calls":16,"redis_duration_s":0.006447,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003748,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001992,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000707,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058964,"duration_s":0.145939,"completed_at":"2026-11-10T11:08:18.388Z","load_balancing_strategy":"primary","db_duration_s":0.02851,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1951gitlab # [ 557.906000] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:18.391Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"107e666ac3002140667520fa","created_at":"2026-11-10T11:06:25.687Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-e15d7fc8a5334cc4","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:25.697Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-107e666ac3002140667520fa: start","job_status":"start","queue_duration_s":112.69406,"scheduling_latency_s":112.694095}1952gitlab # [ 557.977694] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:18.529Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"107e666ac3002140667520fa","created_at":"2026-11-10T11:06:25.687Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-e15d7fc8a5334cc4","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:25.697Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-107e666ac3002140667520fa: done: 0.137878 sec","job_status":"done","queue_duration_s":112.69406,"scheduling_latency_s":112.694095,"redis_calls":15,"redis_duration_s":0.005220000000000001,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001079,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002011,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006809,"duration_s":0.137878,"completed_at":"2026-11-10T11:08:18.528Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 558.048932] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:18.532Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0e5c9addf5a412b30d97a86c","created_at":"2026-11-10T11:06:26.081Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-e15d7fc8a5334cc4","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:26.091Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0e5c9addf5a412b30d97a86c: start","job_status":"start","queue_duration_s":112.440249,"scheduling_latency_s":112.440328}1954gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1955gitlab # [ 558.255577] gitaly[923]: time="2026-11-10T11:08:18.807Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:18.806" grpc.time_ms=0.561 pid=923 span.kind=server system=grpc1956gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.05 seconds)1957gitlab # [ 558.271905] nixbot[4291]: {"ts": "2026-11-10T11:08:18.812Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57172 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1958gitlab # [ 558.288699] gitaly[923]: time="2026-11-10T11:08:18.840Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:18.836" grpc.time_ms=3.72 pid=923 span.kind=server system=grpc1959gitlab # [ 558.326526] gitaly[923]: time="2026-11-10T11:08:18.878Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:18.877" grpc.time_ms=0.619 pid=923 span.kind=server system=grpc1960gitlab # [ 558.340590] gitaly[923]: time="2026-11-10T11:08:18.890Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P diskcache=74f1a247-573f-4dea-a873-82abd55b8862 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:08:48.566" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:18.767" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961gitlab # [ 558.351265] gitaly[923]: time="2026-11-10T11:08:18.891Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=126 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7488 command.oublock=0 command.real_time_ms=116 command.spawn_token_fork_ms=1 command.system_time_ms=58 command.user_time_ms=68 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:08:48.566" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:18.767" grpc.time_ms=123.318 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 558.548535] gitaly[923]: time="2026-11-10T11:08:19.100Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T11:08:49.098" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:19.099" grpc.time_ms=1.164 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 558.608724] gitaly[923]: time="2026-11-10T11:08:19.160Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6H0RE01RY4921EPXE27P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:49.158" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:19.159" grpc.time_ms=1.253 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1964gitlab # [ 558.640377] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.191Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0e5c9addf5a412b30d97a86c","created_at":"2026-11-10T11:06:26.081Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-e15d7fc8a5334cc4","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T11:06:26.091Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0e5c9addf5a412b30d97a86c: done: 0.659361 sec","job_status":"done","queue_duration_s":112.440249,"scheduling_latency_s":112.440328,"gitaly_calls":3,"gitaly_duration_s":0.135695,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.035655,"redis_read_bytes":458,"redis_write_bytes":27997,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002178,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001746,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.012875,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005216,"redis_queues_read_bytes":37,"redis_queues_write_bytes":14491,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.007155,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000916,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000588,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002803,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.033,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.04,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.042,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.420046,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.5565656,"duration_s":0.659361,"completed_at":"2026-11-10T11:08:19.191Z","load_balancing_strategy":"primary","db_duration_s":0.101698,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1965gitlab # [ 558.717793] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.193Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5599f99b1c2db2684ebacd4a272932f856d9fe29 92d9e381fcb2582c86b3e57663b1d0593611d0fe refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ef3dd6091a3633f7266839f5","created_at":"2026-11-10T11:08:12.272Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-715a29446562486f","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:12.278Z","job_size_bytes":185,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ef3dd6091a3633f7266839f5: start","job_status":"start","queue_duration_s":6.915885,"scheduling_latency_s":6.915918}1966gitlab # [ 558.736179] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.212Z","message":"changes: \"5599f99b1c2db2684ebacd4a272932f856d9fe29 92d9e381fcb2582c86b3e57663b1d0593611d0fe refs/heads/master\\n\""}1967gitlab # [ 558.739104] gitaly[923]: time="2026-11-10T11:08:19.252Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:29.069" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:19.239" grpc.time_ms=12.305 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 558.753277] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.279Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8ad0bfca7063e32bb5b21575","created_at":"2026-11-10T11:08:19.276Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-680e9f258438476c","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"01a0f16a6664acffb54e55c8","job_size_bytes":46,"pid":4185,"job_status":"deduplicated","message":"ProjectCacheWorker JID-8ad0bfca7063e32bb5b21575: deduplicated: until executing","deduplication.type":"until executing"}1969gitlab # [ 558.768939] gitaly[923]: time="2026-11-10T11:08:19.272Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T11:08:29.089" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:19.260" grpc.time_ms=12.408 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1970gitlab # [ 558.782991] gitaly[923]: time="2026-11-10T11:08:19.313Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T11:08:29.130" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:19.300" grpc.time_ms=12.577 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971gitlab # [ 558.800967] gitaly[923]: time="2026-11-10T11:08:19.334Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-10T11:08:49.120" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:19.320" grpc.time_ms=13.907 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 558.817230] gitaly[923]: time="2026-11-10T11:08:19.361Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T11:08:29.043" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T11:08:19.353" grpc.time_ms=7.8 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 558.842003] gitaly[923]: time="2026-11-10T11:08:19.394Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:29.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:19.380" grpc.time_ms=13.188 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab # [ 559.024397] gitaly[923]: time="2026-11-10T11:08:19.576Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T11:08:29.264" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:19.574" grpc.time_ms=1.645 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab # [ 559.062091] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.613Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5599f99b1c2db2684ebacd4a272932f856d9fe29 92d9e381fcb2582c86b3e57663b1d0593611d0fe refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ef3dd6091a3633f7266839f5","created_at":"2026-11-10T11:08:12.272Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-715a29446562486f","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:12.278Z","job_size_bytes":185,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ef3dd6091a3633f7266839f5: done: 0.419182 sec","job_status":"done","queue_duration_s":6.915885,"scheduling_latency_s":6.915918,"gitaly_calls":7,"gitaly_duration_s":0.100746,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.033937999999999996,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.003513,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.00118,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005938,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.010655,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.012652,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.049,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.049,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.276854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00141518,"exclusive_lock_hold_duration_s":0.02709564,"duration_s":0.419182,"completed_at":"2026-11-10T11:08:19.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038681,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1976gitlab # [ 559.139266] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"022390ee2e354ef46345a647","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-022390ee2e354ef46345a647: start","job_status":"start","queue_duration_s":4.078393,"scheduling_latency_s":4.078435}1977gitlab # [ 559.153300] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"022390ee2e354ef46345a647","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-022390ee2e354ef46345a647: done: 0.044154 sec","job_status":"done","queue_duration_s":4.078393,"scheduling_latency_s":4.078435,"redis_calls":15,"redis_duration_s":0.004757,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000738,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002488,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044154,"completed_at":"2026-11-10T11:08:19.659Z","load_balancing_strategy":"primary","db_duration_s":0.001005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 559.196293] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165269c54f8b7ae39e049da7","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165269c54f8b7ae39e049da7: start","job_status":"start","queue_duration_s":4.125187,"scheduling_latency_s":4.12523}1979gitlab # [ 559.210361] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165269c54f8b7ae39e049da7","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165269c54f8b7ae39e049da7: done: 0.028201 sec","job_status":"done","queue_duration_s":4.125187,"scheduling_latency_s":4.12523,"redis_calls":13,"redis_duration_s":0.003219,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001252,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028201,"completed_at":"2026-11-10T11:08:19.690Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980gitlab # [ 559.251627] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f29f31b9a6bbc2dd0fc7fa1","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f29f31b9a6bbc2dd0fc7fa1: start","job_status":"start","queue_duration_s":4.155603,"scheduling_latency_s":4.155636}1981gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'1982gitlab # [ 559.265947] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f29f31b9a6bbc2dd0fc7fa1","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f29f31b9a6bbc2dd0fc7fa1: done: 0.024196 sec","job_status":"done","queue_duration_s":4.155603,"scheduling_latency_s":4.155636,"redis_calls":13,"redis_duration_s":0.00277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024196,"completed_at":"2026-11-10T11:08:19.717Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1983gitlab # [ 559.309578] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa0cae010cdf0d286d81996","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa0cae010cdf0d286d81996: start","job_status":"start","queue_duration_s":4.181792,"scheduling_latency_s":4.181825}1984gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.08 seconds)1985gitlab # [ 559.323694] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa0cae010cdf0d286d81996","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa0cae010cdf0d286d81996: done: 0.023814 sec","job_status":"done","queue_duration_s":4.181792,"scheduling_latency_s":4.181825,"redis_calls":13,"redis_duration_s":0.003104,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023814,"completed_at":"2026-11-10T11:08:19.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1986gitlab # [ 559.366867] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be7d057fa21990258cb1317e","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be7d057fa21990258cb1317e: start","job_status":"start","queue_duration_s":4.20721,"scheduling_latency_s":4.207236}1987gitlab # [ 559.380198] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be7d057fa21990258cb1317e","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be7d057fa21990258cb1317e: done: 0.018918 sec","job_status":"done","queue_duration_s":4.20721,"scheduling_latency_s":4.207236,"redis_calls":13,"redis_duration_s":0.0025310000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018918,"completed_at":"2026-11-10T11:08:19.763Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1988gitlab # [ 559.427416] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d51e92dd8a7d0b79f5a74a75","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d51e92dd8a7d0b79f5a74a75: start","job_status":"start","queue_duration_s":4.22793,"scheduling_latency_s":4.227958}1989gitlab # [ 559.441040] nixbot[4291]: {"ts": "2026-11-10T11:08:19.894Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57176 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}1990gitlab # [ 559.443930] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d51e92dd8a7d0b79f5a74a75","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d51e92dd8a7d0b79f5a74a75: done: 0.019791 sec","job_status":"done","queue_duration_s":4.22793,"scheduling_latency_s":4.227958,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019791,"completed_at":"2026-11-10T11:08:19.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 559.485328] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39d1133386e138b0330febd0","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39d1133386e138b0330febd0: start","job_status":"start","queue_duration_s":4.249453,"scheduling_latency_s":4.249478}1992gitlab # [ 559.498376] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39d1133386e138b0330febd0","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39d1133386e138b0330febd0: done: 0.020346 sec","job_status":"done","queue_duration_s":4.249453,"scheduling_latency_s":4.249478,"redis_calls":13,"redis_duration_s":0.002571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001652,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020346,"completed_at":"2026-11-10T11:08:19.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1993gitlab # [ 559.539574] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71ef48ecf967b50f37f29f43","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71ef48ecf967b50f37f29f43: start","job_status":"start","queue_duration_s":4.271612,"scheduling_latency_s":4.271642}1994gitlab # [ 559.552826] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71ef48ecf967b50f37f29f43","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71ef48ecf967b50f37f29f43: done: 0.020412 sec","job_status":"done","queue_duration_s":4.271612,"scheduling_latency_s":4.271642,"redis_calls":13,"redis_duration_s":0.00262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001713,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020412,"completed_at":"2026-11-10T11:08:19.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1995gitlab # [ 559.593524] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c316b5ccad4e4da87c4a709","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c316b5ccad4e4da87c4a709: start","job_status":"start","queue_duration_s":4.293741,"scheduling_latency_s":4.293768}1996gitlab # [ 559.606524] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c316b5ccad4e4da87c4a709","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c316b5ccad4e4da87c4a709: done: 0.020976 sec","job_status":"done","queue_duration_s":4.293741,"scheduling_latency_s":4.293768,"redis_calls":13,"redis_duration_s":0.003223,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020976,"completed_at":"2026-11-10T11:08:19.851Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1997gitlab # [ 559.648988] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9404c462e60183f404b3641","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9404c462e60183f404b3641: start","job_status":"start","queue_duration_s":4.32094,"scheduling_latency_s":4.320977}1998gitlab # [ 559.661861] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9404c462e60183f404b3641","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9404c462e60183f404b3641: done: 0.031342 sec","job_status":"done","queue_duration_s":4.32094,"scheduling_latency_s":4.320977,"redis_calls":13,"redis_duration_s":0.005241,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001746,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031342,"completed_at":"2026-11-10T11:08:19.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999gitlab # [ 559.702591] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a2f0832da990efdfb305308","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a2f0832da990efdfb305308: start","job_status":"start","queue_duration_s":4.354325,"scheduling_latency_s":4.354357}2000gitlab # [ 559.715666] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a2f0832da990efdfb305308","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a2f0832da990efdfb305308: done: 0.02759 sec","job_status":"done","queue_duration_s":4.354325,"scheduling_latency_s":4.354357,"redis_calls":13,"redis_duration_s":0.003066,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02759,"completed_at":"2026-11-10T11:08:19.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001gitlab # [ 559.757190] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5496d30dc47efe15462d236","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5496d30dc47efe15462d236: start","job_status":"start","queue_duration_s":4.384194,"scheduling_latency_s":4.384228}2002gitlab # [ 559.770193] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5496d30dc47efe15462d236","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5496d30dc47efe15462d236: done: 0.024935 sec","job_status":"done","queue_duration_s":4.384194,"scheduling_latency_s":4.384228,"redis_calls":13,"redis_duration_s":0.0030679999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024935,"completed_at":"2026-11-10T11:08:19.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab # [ 559.815871] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e36f23dd8c8cadff4f12e59","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e36f23dd8c8cadff4f12e59: start","job_status":"start","queue_duration_s":4.411244,"scheduling_latency_s":4.411278}2004gitlab # [ 559.830088] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e36f23dd8c8cadff4f12e59","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e36f23dd8c8cadff4f12e59: done: 0.024079 sec","job_status":"done","queue_duration_s":4.411244,"scheduling_latency_s":4.411278,"redis_calls":13,"redis_duration_s":0.0029100000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024079,"completed_at":"2026-11-10T11:08:19.972Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2005gitlab # [ 559.871374] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0ed61d2f4d67597b457baff","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0ed61d2f4d67597b457baff: start","job_status":"start","queue_duration_s":4.437294,"scheduling_latency_s":4.437325}2006gitlab # [ 559.884587] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:19.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0ed61d2f4d67597b457baff","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0ed61d2f4d67597b457baff: done: 0.02387 sec","job_status":"done","queue_duration_s":4.437294,"scheduling_latency_s":4.437325,"redis_calls":13,"redis_duration_s":0.00268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000992,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001688,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02387,"completed_at":"2026-11-10T11:08:19.998Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007gitlab # [ 559.925133] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f25c78e26b47b545bf48656f","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f25c78e26b47b545bf48656f: start","job_status":"start","queue_duration_s":4.463085,"scheduling_latency_s":4.463116}2008gitlab # [ 559.938064] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f25c78e26b47b545bf48656f","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f25c78e26b47b545bf48656f: done: 0.023658 sec","job_status":"done","queue_duration_s":4.463085,"scheduling_latency_s":4.463116,"redis_calls":13,"redis_duration_s":0.002656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001661,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023658,"completed_at":"2026-11-10T11:08:20.024Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2009gitlab # [ 559.979070] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c9bc2c0dec7d60f43d6fb1b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c9bc2c0dec7d60f43d6fb1b: start","job_status":"start","queue_duration_s":4.489031,"scheduling_latency_s":4.489062}2010gitlab # [ 559.992131] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c9bc2c0dec7d60f43d6fb1b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c9bc2c0dec7d60f43d6fb1b: done: 0.025233 sec","job_status":"done","queue_duration_s":4.489031,"scheduling_latency_s":4.489062,"redis_calls":13,"redis_duration_s":0.003034,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025233,"completed_at":"2026-11-10T11:08:20.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2011gitlab # [ 560.034215] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e999459e76f175369a2b508","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e999459e76f175369a2b508: start","job_status":"start","queue_duration_s":4.516269,"scheduling_latency_s":4.516299}2012gitlab # [ 560.047283] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e999459e76f175369a2b508","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e999459e76f175369a2b508: done: 0.023365 sec","job_status":"done","queue_duration_s":4.516269,"scheduling_latency_s":4.516299,"redis_calls":13,"redis_duration_s":0.002718,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000991,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001727,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023365,"completed_at":"2026-11-10T11:08:20.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2013gitlab # [ 560.088436] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"977cf79005fa89aefc7727a2","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-977cf79005fa89aefc7727a2: start","job_status":"start","queue_duration_s":4.541616,"scheduling_latency_s":4.541646}2014gitlab # [ 560.101170] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"977cf79005fa89aefc7727a2","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-977cf79005fa89aefc7727a2: done: 0.023071 sec","job_status":"done","queue_duration_s":4.541616,"scheduling_latency_s":4.541646,"redis_calls":13,"redis_duration_s":0.00273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001721,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023071,"completed_at":"2026-11-10T11:08:20.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2015gitlab # [ 560.141851] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50486a52764844dd10635dfa","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50486a52764844dd10635dfa: start","job_status":"start","queue_duration_s":4.566579,"scheduling_latency_s":4.566608}2016gitlab # [ 560.154764] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50486a52764844dd10635dfa","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50486a52764844dd10635dfa: done: 0.023272 sec","job_status":"done","queue_duration_s":4.566579,"scheduling_latency_s":4.566608,"redis_calls":13,"redis_duration_s":0.002928,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023272,"completed_at":"2026-11-10T11:08:20.127Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab # [ 560.195713] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fcd767d361d5b6c2190f802","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fcd767d361d5b6c2190f802: start","job_status":"start","queue_duration_s":4.592824,"scheduling_latency_s":4.59286}2018gitlab # [ 560.208886] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fcd767d361d5b6c2190f802","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fcd767d361d5b6c2190f802: done: 0.022004 sec","job_status":"done","queue_duration_s":4.592824,"scheduling_latency_s":4.59286,"redis_calls":13,"redis_duration_s":0.002783,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001714,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022004,"completed_at":"2026-11-10T11:08:20.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2019gitlab # [ 560.251169] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"996bd8244e09735d18536cf7","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-996bd8244e09735d18536cf7: start","job_status":"start","queue_duration_s":4.616356,"scheduling_latency_s":4.616381}2020gitlab # [ 560.264661] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"996bd8244e09735d18536cf7","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-996bd8244e09735d18536cf7: done: 0.017413 sec","job_status":"done","queue_duration_s":4.616356,"scheduling_latency_s":4.616381,"redis_calls":13,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001524,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017413,"completed_at":"2026-11-10T11:08:20.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2021gitlab # [ 560.306029] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feb05cb5ed199ec5c0974166","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feb05cb5ed199ec5c0974166: start","job_status":"start","queue_duration_s":4.635371,"scheduling_latency_s":4.635397}2022gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'2023gitlab # [ 560.319278] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feb05cb5ed199ec5c0974166","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feb05cb5ed199ec5c0974166: done: 0.018116 sec","job_status":"done","queue_duration_s":4.635371,"scheduling_latency_s":4.635397,"redis_calls":13,"redis_duration_s":0.002541,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001687,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018116,"completed_at":"2026-11-10T11:08:20.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 560.360966] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8539b873cd7f8e962023d3f7","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8539b873cd7f8e962023d3f7: start","job_status":"start","queue_duration_s":4.655101,"scheduling_latency_s":4.655126}2025gitlab # [ 560.373604] gitaly[923]: time="2026-11-10T11:08:20.846Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-10T17:08:20.839" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:20.840" grpc.time_ms=5.866 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2026gitlab # [ 560.384860] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8539b873cd7f8e962023d3f7","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8539b873cd7f8e962023d3f7: done: 0.018292 sec","job_status":"done","queue_duration_s":4.655101,"scheduling_latency_s":4.655126,"redis_calls":13,"redis_duration_s":0.002373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001525,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018292,"completed_at":"2026-11-10T11:08:20.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2027gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.07 seconds)2028gitlab # [ 560.426437] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee1655ff2c4e614d11e20f4","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee1655ff2c4e614d11e20f4: start","job_status":"start","queue_duration_s":4.675052,"scheduling_latency_s":4.675077}2029gitlab # [ 560.439690] gitaly[923]: time="2026-11-10T11:08:20.870Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T11:08:50.658" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:20.858" grpc.time_ms=11.464 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2030gitlab # [ 560.455542] nixbot[4291]: {"ts": "2026-11-10T11:08:20.967Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57190 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}2031gitlab # [ 560.458845] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee1655ff2c4e614d11e20f4","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee1655ff2c4e614d11e20f4: done: 0.018146 sec","job_status":"done","queue_duration_s":4.675052,"scheduling_latency_s":4.675077,"redis_calls":13,"redis_duration_s":0.002332,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001517,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018146,"completed_at":"2026-11-10T11:08:20.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2032gitlab # [ 560.500601] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce38ad2b07500df1158eddb9","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce38ad2b07500df1158eddb9: start","job_status":"start","queue_duration_s":4.695212,"scheduling_latency_s":4.695239}2033gitlab # [ 560.513712] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce38ad2b07500df1158eddb9","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce38ad2b07500df1158eddb9: done: 0.018519 sec","job_status":"done","queue_duration_s":4.695212,"scheduling_latency_s":4.695239,"redis_calls":13,"redis_duration_s":0.002457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001583,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018519,"completed_at":"2026-11-10T11:08:20.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034gitlab # [ 560.555072] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afaffa0738a0897acda65fb8","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afaffa0738a0897acda65fb8: start","job_status":"start","queue_duration_s":4.715505,"scheduling_latency_s":4.715532}2035gitlab # [ 560.568318] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afaffa0738a0897acda65fb8","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afaffa0738a0897acda65fb8: done: 0.018592 sec","job_status":"done","queue_duration_s":4.715505,"scheduling_latency_s":4.715532,"redis_calls":13,"redis_duration_s":0.002615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018592,"completed_at":"2026-11-10T11:08:20.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2036gitlab # [ 560.610293] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbf454f562cf3720c5151f31","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbf454f562cf3720c5151f31: start","job_status":"start","queue_duration_s":4.735772,"scheduling_latency_s":4.735797}2037gitlab # [ 560.623434] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbf454f562cf3720c5151f31","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbf454f562cf3720c5151f31: done: 0.018789 sec","job_status":"done","queue_duration_s":4.735772,"scheduling_latency_s":4.735797,"redis_calls":13,"redis_duration_s":0.002611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001695,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018789,"completed_at":"2026-11-10T11:08:20.291Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038gitlab # [ 560.665929] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5c88abb30488d1dbf933516","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5c88abb30488d1dbf933516: start","job_status":"start","queue_duration_s":4.756443,"scheduling_latency_s":4.756468}2039gitlab # [ 560.679018] gitaly[923]: time="2026-11-10T11:08:21.160Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ diskcache=cef4147f-f574-4a25-8037-3fbb24e829c7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T17:08:21.127" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:21.127" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040gitlab # [ 560.689676] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5c88abb30488d1dbf933516","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5c88abb30488d1dbf933516: done: 0.018799 sec","job_status":"done","queue_duration_s":4.756443,"scheduling_latency_s":4.756468,"redis_calls":13,"redis_duration_s":0.002427,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001594,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018799,"completed_at":"2026-11-10T11:08:20.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2041gitlab # [ 560.732699] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b2f08a8f6bc969d51d2b6b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b2f08a8f6bc969d51d2b6b: start","job_status":"start","queue_duration_s":4.777132,"scheduling_latency_s":4.777157}2042gitlab # [ 560.745938] gitaly[923]: time="2026-11-10T11:08:21.160Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1177 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=3 command.system_time_ms=15 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T17:08:21.127" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:21.127" grpc.time_ms=32.319 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2043gitlab # [ 560.762760] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b2f08a8f6bc969d51d2b6b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b2f08a8f6bc969d51d2b6b: done: 0.019017 sec","job_status":"done","queue_duration_s":4.777132,"scheduling_latency_s":4.777157,"redis_calls":13,"redis_duration_s":0.002578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001706,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019017,"completed_at":"2026-11-10T11:08:20.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2044gitlab # [ 560.804690] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c9cb3b0a31cfe155e37a93e","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c9cb3b0a31cfe155e37a93e: start","job_status":"start","queue_duration_s":4.797844,"scheduling_latency_s":4.797867}2045gitlab # [ 560.817951] gitaly[923]: time="2026-11-10T11:08:21.295Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T11:08:51.083" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:21.283" grpc.time_ms=11.797 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2046gitlab # [ 560.832099] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c9cb3b0a31cfe155e37a93e","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c9cb3b0a31cfe155e37a93e: done: 0.019631 sec","job_status":"done","queue_duration_s":4.797844,"scheduling_latency_s":4.797867,"redis_calls":13,"redis_duration_s":0.003018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001327,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001691,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019631,"completed_at":"2026-11-10T11:08:20.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047gitlab # [ 560.874815] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b142ed1f02e593e808422c8","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b142ed1f02e593e808422c8: start","job_status":"start","queue_duration_s":4.819356,"scheduling_latency_s":4.819384}2048gitlab # [ 560.888239] gitaly[923]: time="2026-11-10T11:08:21.376Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:51.174" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:21.374" grpc.time_ms=1.045 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2049gitlab # [ 560.901350] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b142ed1f02e593e808422c8","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b142ed1f02e593e808422c8: done: 0.01958 sec","job_status":"done","queue_duration_s":4.819356,"scheduling_latency_s":4.819384,"redis_calls":13,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001799,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01958,"completed_at":"2026-11-10T11:08:20.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2050gitlab # [ 560.943174] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d3bd5770727ca407c5e71e5","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d3bd5770727ca407c5e71e5: start","job_status":"start","queue_duration_s":4.840671,"scheduling_latency_s":4.840699}2051gitlab # [ 560.956613] gitaly[923]: time="2026-11-10T11:08:21.392Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T11:08:31.069" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:21.380" grpc.time_ms=11.618 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2052gitlab # [ 560.970570] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d3bd5770727ca407c5e71e5","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d3bd5770727ca407c5e71e5: done: 0.020002 sec","job_status":"done","queue_duration_s":4.840671,"scheduling_latency_s":4.840699,"redis_calls":13,"redis_duration_s":0.002782,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020002,"completed_at":"2026-11-10T11:08:20.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053gitlab # [ 561.012524] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33fe785d104be107a1caf4aa","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33fe785d104be107a1caf4aa: start","job_status":"start","queue_duration_s":4.862324,"scheduling_latency_s":4.862352}2054gitlab # [ 561.025540] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33fe785d104be107a1caf4aa","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33fe785d104be107a1caf4aa: done: 0.019507 sec","job_status":"done","queue_duration_s":4.862324,"scheduling_latency_s":4.862352,"redis_calls":13,"redis_duration_s":0.0024950000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019507,"completed_at":"2026-11-10T11:08:20.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055gitlab # [ 561.067467] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79353b1f592ce33842bc28ab","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79353b1f592ce33842bc28ab: start","job_status":"start","queue_duration_s":4.883594,"scheduling_latency_s":4.883622}2056gitlab # [ 561.081912] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79353b1f592ce33842bc28ab","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79353b1f592ce33842bc28ab: done: 0.020109 sec","job_status":"done","queue_duration_s":4.883594,"scheduling_latency_s":4.883622,"redis_calls":13,"redis_duration_s":0.003149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002235,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020109,"completed_at":"2026-11-10T11:08:20.440Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057gitlab # [ 561.123435] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2af015b087c7529f23733acc","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2af015b087c7529f23733acc: start","job_status":"start","queue_duration_s":4.905391,"scheduling_latency_s":4.905416}2058gitlab # [ 561.136860] gitaly[923]: time="2026-11-10T11:08:21.605Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T11:08:51.404" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:21.604" grpc.time_ms=1.297 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2059gitlab # [ 561.150270] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2af015b087c7529f23733acc","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2af015b087c7529f23733acc: done: 0.020062 sec","job_status":"done","queue_duration_s":4.905391,"scheduling_latency_s":4.905416,"redis_calls":13,"redis_duration_s":0.002591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001721,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020062,"completed_at":"2026-11-10T11:08:20.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2060gitlab # [ 561.191989] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b626459d057a768d0002216","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b626459d057a768d0002216: start","job_status":"start","queue_duration_s":4.927189,"scheduling_latency_s":4.927216}2061gitlab # [ 561.204925] gitaly[923]: time="2026-11-10T11:08:21.615Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=45 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T11:08:51.411" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:21.611" grpc.time_ms=3.45 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2062gitlab # [ 561.221873] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b626459d057a768d0002216","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b626459d057a768d0002216: done: 0.018801 sec","job_status":"done","queue_duration_s":4.927189,"scheduling_latency_s":4.927216,"redis_calls":13,"redis_duration_s":0.002508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001645,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018801,"completed_at":"2026-11-10T11:08:20.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 561.263213] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab243dcacb5a04289a86b2cf","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab243dcacb5a04289a86b2cf: start","job_status":"start","queue_duration_s":4.947668,"scheduling_latency_s":4.947692}2064gitlab # [ 561.276994] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab243dcacb5a04289a86b2cf","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab243dcacb5a04289a86b2cf: done: 0.017833 sec","job_status":"done","queue_duration_s":4.947668,"scheduling_latency_s":4.947692,"redis_calls":13,"redis_duration_s":0.002544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017833,"completed_at":"2026-11-10T11:08:20.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 561.319582] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02c4e50fe1144620002e1ce4","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02c4e50fe1144620002e1ce4: start","job_status":"start","queue_duration_s":4.967066,"scheduling_latency_s":4.967091}2066gitlab # [ 561.332842] gitaly[923]: time="2026-11-10T11:08:21.786Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:31.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:21.774" grpc.time_ms=12.344 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 561.346541] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02c4e50fe1144620002e1ce4","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02c4e50fe1144620002e1ce4: done: 0.01806 sec","job_status":"done","queue_duration_s":4.967066,"scheduling_latency_s":4.967091,"redis_calls":13,"redis_duration_s":0.002299,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001501,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01806,"completed_at":"2026-11-10T11:08:20.522Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068gitlab # [ 561.387786] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba5a887d7637b39823e0b87","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba5a887d7637b39823e0b87: start","job_status":"start","queue_duration_s":4.986765,"scheduling_latency_s":4.986915}2069gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'2070gitlab # [ 561.401149] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba5a887d7637b39823e0b87","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba5a887d7637b39823e0b87: done: 0.019015 sec","job_status":"done","queue_duration_s":4.986765,"scheduling_latency_s":4.986915,"redis_calls":13,"redis_duration_s":0.002537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001663,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019015,"completed_at":"2026-11-10T11:08:20.543Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071gitlab # [ 561.441463] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2d058f9341dae59f256453b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2d058f9341dae59f256453b: start","job_status":"start","queue_duration_s":5.007583,"scheduling_latency_s":5.007607}2072gitlab # [ 561.454334] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2d058f9341dae59f256453b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2d058f9341dae59f256453b: done: 0.01878 sec","job_status":"done","queue_duration_s":5.007583,"scheduling_latency_s":5.007607,"redis_calls":13,"redis_duration_s":0.002569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001669,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01878,"completed_at":"2026-11-10T11:08:20.563Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2073gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.07 seconds)2074gitlab # [ 561.495986] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e07fa6649bb3ad46cba8fd8","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e07fa6649bb3ad46cba8fd8: start","job_status":"start","queue_duration_s":5.028331,"scheduling_latency_s":5.028362}2075gitlab # [ 561.510196] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e07fa6649bb3ad46cba8fd8","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e07fa6649bb3ad46cba8fd8: done: 0.020232 sec","job_status":"done","queue_duration_s":5.028331,"scheduling_latency_s":5.028362,"redis_calls":13,"redis_duration_s":0.0029430000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020232,"completed_at":"2026-11-10T11:08:20.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 561.551632] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63610b900bd10b131f4696aa","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63610b900bd10b131f4696aa: start","job_status":"start","queue_duration_s":5.050537,"scheduling_latency_s":5.050564}2077gitlab # [ 561.565252] nixbot[4291]: {"ts": "2026-11-10T11:08:22.037Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35562 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}2078gitlab # [ 561.568185] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63610b900bd10b131f4696aa","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63610b900bd10b131f4696aa: done: 0.020025 sec","job_status":"done","queue_duration_s":5.050537,"scheduling_latency_s":5.050564,"redis_calls":13,"redis_duration_s":0.003112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001242,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020025,"completed_at":"2026-11-10T11:08:20.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2079gitlab # [ 561.610383] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0d87c87ad0a20252bc5cf7a","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0d87c87ad0a20252bc5cf7a: start","job_status":"start","queue_duration_s":5.072508,"scheduling_latency_s":5.072537}2080gitlab # [ 561.623757] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0d87c87ad0a20252bc5cf7a","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0d87c87ad0a20252bc5cf7a: done: 0.019984 sec","job_status":"done","queue_duration_s":5.072508,"scheduling_latency_s":5.072537,"redis_calls":13,"redis_duration_s":0.0027819999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019984,"completed_at":"2026-11-10T11:08:20.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 561.666055] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"089141086e99475c574f1562","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-089141086e99475c574f1562: start","job_status":"start","queue_duration_s":5.094304,"scheduling_latency_s":5.09433}2082gitlab # [ 561.679424] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"089141086e99475c574f1562","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-089141086e99475c574f1562: done: 0.019828 sec","job_status":"done","queue_duration_s":5.094304,"scheduling_latency_s":5.09433,"redis_calls":13,"redis_duration_s":0.002815,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019828,"completed_at":"2026-11-10T11:08:20.651Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2083gitlab # [ 561.721889] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb9dbffe64cac7809d10c6e0","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb9dbffe64cac7809d10c6e0: start","job_status":"start","queue_duration_s":5.115715,"scheduling_latency_s":5.11574}2084gitlab # [ 561.735030] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb9dbffe64cac7809d10c6e0","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb9dbffe64cac7809d10c6e0: done: 0.019527 sec","job_status":"done","queue_duration_s":5.115715,"scheduling_latency_s":5.11574,"redis_calls":13,"redis_duration_s":0.002627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019527,"completed_at":"2026-11-10T11:08:20.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 561.776820] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b686e81f1c0c018435e5e7a4","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b686e81f1c0c018435e5e7a4: start","job_status":"start","queue_duration_s":5.13685,"scheduling_latency_s":5.136875}2086gitlab # [ 561.789885] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b686e81f1c0c018435e5e7a4","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b686e81f1c0c018435e5e7a4: done: 0.019514 sec","job_status":"done","queue_duration_s":5.13685,"scheduling_latency_s":5.136875,"redis_calls":13,"redis_duration_s":0.002558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001681,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019514,"completed_at":"2026-11-10T11:08:20.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 561.831353] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ee8f5047fb7f306f4c85318","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ee8f5047fb7f306f4c85318: start","job_status":"start","queue_duration_s":5.15798,"scheduling_latency_s":5.158005}2088gitlab # [ 561.844747] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ee8f5047fb7f306f4c85318","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ee8f5047fb7f306f4c85318: done: 0.019572 sec","job_status":"done","queue_duration_s":5.15798,"scheduling_latency_s":5.158005,"redis_calls":13,"redis_duration_s":0.002462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019572,"completed_at":"2026-11-10T11:08:20.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2089gitlab # [ 561.885921] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"111a35bddec343d5ad1f5b5b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-111a35bddec343d5ad1f5b5b: start","job_status":"start","queue_duration_s":5.179294,"scheduling_latency_s":5.179319}2090gitlab # [ 561.899058] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"111a35bddec343d5ad1f5b5b","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-111a35bddec343d5ad1f5b5b: done: 0.019315 sec","job_status":"done","queue_duration_s":5.179294,"scheduling_latency_s":5.179319,"redis_calls":13,"redis_duration_s":0.002933,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019315,"completed_at":"2026-11-10T11:08:20.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091gitlab # [ 561.941526] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a451a5ef45f9b64f8064b77c","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a451a5ef45f9b64f8064b77c: start","job_status":"start","queue_duration_s":5.200592,"scheduling_latency_s":5.200618}2092gitlab # [ 561.954562] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a451a5ef45f9b64f8064b77c","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a451a5ef45f9b64f8064b77c: done: 0.019964 sec","job_status":"done","queue_duration_s":5.200592,"scheduling_latency_s":5.200618,"redis_calls":13,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019964,"completed_at":"2026-11-10T11:08:20.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2093gitlab # [ 561.996340] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50e276d8e979efa9e31305d1","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50e276d8e979efa9e31305d1: start","job_status":"start","queue_duration_s":5.222295,"scheduling_latency_s":5.222319}2094gitlab # [ 562.009443] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50e276d8e979efa9e31305d1","created_at":"2026-11-10T11:08:15.420Z","trace_propagation_headers":{"sentry-trace":"b262468473b448d1bb3cb79ecbbaed2b-dea84d96e8144cd7","baggage":"sentry-trace_id=b262468473b448d1bb3cb79ecbbaed2b,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f391501ab784309c7e46e4615ec7fc5d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:15.537Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50e276d8e979efa9e31305d1: done: 0.019239 sec","job_status":"done","queue_duration_s":5.222295,"scheduling_latency_s":5.222319,"redis_calls":13,"redis_duration_s":0.002444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001584,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019239,"completed_at":"2026-11-10T11:08:20.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095gitlab # [ 562.050622] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.781Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"01a0f16a6664acffb54e55c8","created_at":"2026-11-10T11:08:16.687Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:16.700Z","job_size_bytes":46,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-01a0f16a6664acffb54e55c8: start","job_status":"start","queue_duration_s":4.080547,"scheduling_latency_s":4.080574}2096gitlab # [ 562.067510] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.915Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"01a0f16a6664acffb54e55c8","created_at":"2026-11-10T11:08:16.687Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:16.700Z","job_size_bytes":46,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-01a0f16a6664acffb54e55c8: done: 0.134679 sec","job_status":"done","queue_duration_s":4.080547,"scheduling_latency_s":4.080574,"gitaly_calls":2,"gitaly_duration_s":0.023308,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009144,"redis_read_bytes":69,"redis_write_bytes":4879,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001673,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000848,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002077,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003612,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00073736,"duration_s":0.134679,"completed_at":"2026-11-10T11:08:20.915Z","load_balancing_strategy":"primary","db_duration_s":0.016701,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2097gitlab # [ 562.118678] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:20.918Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"68252287530e523491ddb365","created_at":"2026-11-10T11:08:16.729Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:16.738Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-68252287530e523491ddb365: start","job_status":"start","queue_duration_s":4.179434,"scheduling_latency_s":4.179469}2098gitlab # [ 562.141762] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:21.235Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"68252287530e523491ddb365","created_at":"2026-11-10T11:08:16.729Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:16.738Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-68252287530e523491ddb365: done: 0.31722 sec","job_status":"done","queue_duration_s":4.179434,"scheduling_latency_s":4.179469,"gitaly_calls":1,"gitaly_duration_s":0.036038,"redis_calls":15,"redis_duration_s":0.004783,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000662,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.027,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.043,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10600004,"duration_s":0.31722,"completed_at":"2026-11-10T11:08:21.235Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03252,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 562.190872] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:21.238Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9f8c99107b398b6756b0f74c","created_at":"2026-11-10T11:08:16.879Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:16.883Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9f8c99107b398b6756b0f74c: start","job_status":"start","queue_duration_s":4.35453,"scheduling_latency_s":4.354563}2100gitlab # [ 562.207256] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:21.315Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9f8c99107b398b6756b0f74c","created_at":"2026-11-10T11:08:16.879Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:16.883Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9f8c99107b398b6756b0f74c: done: 0.077089 sec","job_status":"done","queue_duration_s":4.35453,"scheduling_latency_s":4.354563,"gitaly_calls":1,"gitaly_duration_s":0.015423,"redis_calls":16,"redis_duration_s":0.005759,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000628,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002471,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.077089,"completed_at":"2026-11-10T11:08:21.315Z","load_balancing_strategy":"primary","db_duration_s":0.004799,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2101gitlab # [ 562.255329] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:21.318Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"ec78306402e179513d8f4bc9","created_at":"2026-11-10T11:08:17.899Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:411fa69d352912917c28e45f00a2ceb86a57d69e28f84523fc77cd53e64f7a21","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:17.912Z","job_size_bytes":467,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-ec78306402e179513d8f4bc9: start","job_status":"start","queue_duration_s":3.405918,"scheduling_latency_s":3.405964}2102gitlab # [ 562.274086] gitaly[923]: time="2026-11-10T11:08:22.807Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=879 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T11:08:52.787" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:22.787" grpc.time_ms=19.492 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2103gitlab # [ 562.290214] gitaly[923]: time="2026-11-10T11:08:22.832Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J6A23Z4JRKVEGMCJXQJHZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T11:08:52.812" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:22.812" grpc.time_ms=19.367 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104gitlab # [ 562.327874] sidekiq-cluster[4185]: {"severity":"WARN","time":"2026-11-10T11:08:22.879Z","class":"CreatePipelineWorker","correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"5599f99b1c2db2684ebacd4a272932f856d9fe29","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"5599f99b1c2db2684ebacd4a272932f856d9fe29","ref":"refs/heads/master"},"retry":0}2105gitlab # [ 562.342168] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:22.893Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"ec78306402e179513d8f4bc9","created_at":"2026-11-10T11:08:17.899Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:411fa69d352912917c28e45f00a2ceb86a57d69e28f84523fc77cd53e64f7a21","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:17.912Z","job_size_bytes":467,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-ec78306402e179513d8f4bc9: done: 1.575382 sec","job_status":"done","queue_duration_s":3.405918,"scheduling_latency_s":3.405964,"gitaly_calls":7,"gitaly_duration_s":0.093909,"redis_calls":51,"redis_duration_s":0.015585,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001241,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.006637,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002413,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003913,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001381,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.407598,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040802,"duration_s":1.575382,"completed_at":"2026-11-10T11:08:22.893Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.082657,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2106gitlab # [ 562.396096] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:22.896Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"8a559a04-0d89-4800-ab12-a10578e76421\"}"],"class":"WebHookWorker","jid":"a749587895087140c90bba99","created_at":"2026-11-10T11:08:18.123Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f6a602d01cada6068cc866a774f0642147fa980cdde0f74c8e1e634e8db32081","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.133Z","job_size_bytes":1751,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a749587895087140c90bba99: start","job_status":"start","queue_duration_s":4.763161,"scheduling_latency_s":4.763204}2107gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'2108gitlab # [ 562.575225] nixbot[4291]: {"ts": "2026-11-10T11:08:23.126Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "480d0e50-f350-49d8-acdc-fae6f573e3ab", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "5599f99b1c2db2684ebacd4a272932f856d9fe29", "pr": null}2109gitlab # [ 562.583022] nixbot[4291]: {"ts": "2026-11-10T11:08:23.133Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35574 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}2110gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.09 seconds)2111gitlab # [ 562.588584] nixbot[4291]: {"ts": "2026-11-10T11:08:23.135Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35570 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}2112gitlab # [ 562.619467] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.170Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"8a559a04-0d89-4800-ab12-a10578e76421\"}"],"class":"WebHookWorker","jid":"a749587895087140c90bba99","created_at":"2026-11-10T11:08:18.123Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-49ec42b5d8974c76","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f6a602d01cada6068cc866a774f0642147fa980cdde0f74c8e1e634e8db32081","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.133Z","job_size_bytes":1751,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a749587895087140c90bba99: done: 0.27373 sec","job_status":"done","queue_duration_s":4.763161,"scheduling_latency_s":4.763204,"redis_calls":19,"redis_duration_s":0.007313,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001523,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000476,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003655,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001659,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.08659962000001542,"cpu_s":0.160256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00084144,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.27373,"completed_at":"2026-11-10T11:08:23.170Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2113gitlab # [ 562.692571] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.174Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"913bf763ccab93b82c6c0461","created_at":"2026-11-10T11:08:18.741Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.749Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-913bf763ccab93b82c6c0461: start","job_status":"start","queue_duration_s":4.425042,"scheduling_latency_s":4.425091}2114gitlab # [ 562.718034] gitaly[923]: time="2026-11-10T11:08:23.270Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=6 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA47ZMR8D90B30J7TWQ76 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:23.254" grpc.time_ms=15.842 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2115gitlab # [ 562.741721] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:23 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1032116gitlab # [ 562.772971] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.324Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"913bf763ccab93b82c6c0461","created_at":"2026-11-10T11:08:18.741Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.749Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-913bf763ccab93b82c6c0461: done: 0.149973 sec","job_status":"done","queue_duration_s":4.425042,"scheduling_latency_s":4.425091,"redis_calls":10,"redis_duration_s":0.003627,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000742,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00082,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.114131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00074986,"duration_s":0.149973,"completed_at":"2026-11-10T11:08:23.324Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028086,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2117gitlab # [ 562.838985] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.326Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2bdcac5db6afbfa95a7b5bb8","created_at":"2026-11-10T11:08:18.899Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.908Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2bdcac5db6afbfa95a7b5bb8: start","job_status":"start","queue_duration_s":4.418763,"scheduling_latency_s":4.418797}2118gitlab # [ 562.857505] gitlab-workhorse[829]: time="2026-11-10T11:08:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672119gitlab # [ 562.859392] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1032120gitlab # [ 562.861507] gitaly[923]: time="2026-11-10T11:08:23.376Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4B9RHE41Z7HGRERQKJB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:23.364" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2121gitlab # [ 562.872043] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.399Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2bdcac5db6afbfa95a7b5bb8","created_at":"2026-11-10T11:08:18.899Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.908Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2bdcac5db6afbfa95a7b5bb8: done: 0.073065 sec","job_status":"done","queue_duration_s":4.418763,"scheduling_latency_s":4.418797,"redis_calls":10,"redis_duration_s":0.003523,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000664,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000689,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048086,"duration_s":0.073065,"completed_at":"2026-11-10T11:08:23.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122gitlab # [ 562.918669] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.402Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ccc14de99d70c1aec784e3c4","created_at":"2026-11-10T11:08:18.930Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.941Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ccc14de99d70c1aec784e3c4: start","job_status":"start","queue_duration_s":4.460889,"scheduling_latency_s":4.460923}2123gitlab # [ 562.935516] gitaly[923]: time="2026-11-10T11:08:23.376Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4B9RHE41Z7HGRERQKJB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:23.364" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2124gitlab # [ 562.951388] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.460Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ccc14de99d70c1aec784e3c4","created_at":"2026-11-10T11:08:18.930Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.941Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ccc14de99d70c1aec784e3c4: done: 0.058274 sec","job_status":"done","queue_duration_s":4.460889,"scheduling_latency_s":4.460923,"redis_calls":10,"redis_duration_s":0.0029909999999999997,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00067,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000663,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039062,"duration_s":0.058274,"completed_at":"2026-11-10T11:08:23.460Z","load_balancing_strategy":"primary","db_duration_s":0.009631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 563.001460] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.465Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2b9f0935e691a2d519a512f5","created_at":"2026-11-10T11:08:18.964Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.975Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2b9f0935e691a2d519a512f5: start","job_status":"start","queue_duration_s":4.489437,"scheduling_latency_s":4.489474}2126gitlab # [ 563.017904] gitlab-workhorse[829]: time="2026-11-10T11:08:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3372127gitlab # [ 563.019785] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 1062128gitlab # [ 563.021929] gitaly[923]: time="2026-11-10T11:08:23.377Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4B9RHE41Z7HGRERQKJB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:23.364" grpc.time_ms=12.145 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2129gitlab # [ 563.036937] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.542Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2b9f0935e691a2d519a512f5","created_at":"2026-11-10T11:08:18.964Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.975Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2b9f0935e691a2d519a512f5: done: 0.077708 sec","job_status":"done","queue_duration_s":4.489437,"scheduling_latency_s":4.489474,"redis_calls":13,"redis_duration_s":0.005294,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001116,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001018,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050578,"duration_s":0.077708,"completed_at":"2026-11-10T11:08:23.542Z","load_balancing_strategy":"primary","db_duration_s":0.016344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130gitlab # [ 563.083866] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.545Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3016c7e30254a885821de79a","created_at":"2026-11-10T11:08:18.977Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.986Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3016c7e30254a885821de79a: start","job_status":"start","queue_duration_s":4.559444,"scheduling_latency_s":4.559481}2131gitlab # [ 563.101480] nixbot[4291]: {"ts": "2026-11-10T11:08:23.532Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2132gitlab # [ 563.103947] nixbot[4291]: {"ts": "2026-11-10T11:08:23.534Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "5599f99b1c2db2684ebacd4a272932f856d9fe29"}2133gitlab # [ 563.106798] gitaly[923]: time="2026-11-10T11:08:23.485Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4EMPP70MBSTCFSE4J89 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:23.462" pid=923 remote_ip="2001:db8:1::1" request_sha=1fe556d2ea010efffadfd9ee9241e9fcbce7da9c response_bytes=337 span.kind=server system=grpc2134gitlab # [ 563.117597] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.627Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3016c7e30254a885821de79a","created_at":"2026-11-10T11:08:18.977Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.986Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3016c7e30254a885821de79a: done: 0.081439 sec","job_status":"done","queue_duration_s":4.559444,"scheduling_latency_s":4.559481,"redis_calls":10,"redis_duration_s":0.00389,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00070052,"duration_s":0.081439,"completed_at":"2026-11-10T11:08:23.626Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135gitlab # [ 563.163872] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.629Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c16ae1074d9f19a73d0f6000","created_at":"2026-11-10T11:08:18.987Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.996Z","job_size_bytes":19,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c16ae1074d9f19a73d0f6000: start","job_status":"start","queue_duration_s":4.633241,"scheduling_latency_s":4.633281}2136gitlab # [ 563.184532] gitaly[923]: time="2026-11-10T11:08:23.485Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1130 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=4 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4EMPP70MBSTCFSE4J89 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:23.462" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2137gitlab # [ 563.198861] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.710Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c16ae1074d9f19a73d0f6000","created_at":"2026-11-10T11:08:18.987Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:18.996Z","job_size_bytes":19,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c16ae1074d9f19a73d0f6000: done: 0.080847 sec","job_status":"done","queue_duration_s":4.633241,"scheduling_latency_s":4.633281,"redis_calls":11,"redis_duration_s":0.018751,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.017315,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069086,"duration_s":0.080847,"completed_at":"2026-11-10T11:08:23.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138gitlab # [ 563.244864] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.713Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"33a865009bb0a3d956f53c26","created_at":"2026-11-10T11:08:19.001Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.010Z","job_size_bytes":66,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-33a865009bb0a3d956f53c26: start","job_status":"start","queue_duration_s":4.702657,"scheduling_latency_s":4.702694}2139gitlab # [ 563.262826] gitaly[923]: time="2026-11-10T11:08:23.486Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1130 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=4 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4EMPP70MBSTCFSE4J89 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:23.462" grpc.time_ms=23.683 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2140gitlab # [ 563.278505] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.794Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"33a865009bb0a3d956f53c26","created_at":"2026-11-10T11:08:19.001Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.010Z","job_size_bytes":66,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-33a865009bb0a3d956f53c26: done: 0.080794 sec","job_status":"done","queue_duration_s":4.702657,"scheduling_latency_s":4.702694,"redis_calls":10,"redis_duration_s":0.005077,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001114,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002015,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005221,"duration_s":0.080794,"completed_at":"2026-11-10T11:08:23.794Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 563.330215] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.797Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"666d1f54b72c3ea621e54cfe","created_at":"2026-11-10T11:08:19.012Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.021Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-666d1f54b72c3ea621e54cfe: start","job_status":"start","queue_duration_s":4.775979,"scheduling_latency_s":4.776017}2142gitlab # [ 563.347777] gitaly[923]: time="2026-11-10T11:08:23.656Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:53.254" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:23.654" grpc.time_ms=0.816 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2143gitlab # [ 563.361439] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:23 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5599f99b1c2db2684ebacd4a272932f856d9fe29 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3242144gitlab # [ 563.364091] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.863Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"666d1f54b72c3ea621e54cfe","created_at":"2026-11-10T11:08:19.012Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.021Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-666d1f54b72c3ea621e54cfe: done: 0.065401 sec","job_status":"done","queue_duration_s":4.775979,"scheduling_latency_s":4.776017,"redis_calls":10,"redis_duration_s":0.002986,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000536,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000674,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058814,"duration_s":0.065401,"completed_at":"2026-11-10T11:08:23.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 563.412799] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.865Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2ea1f740a83ca3cd56952c71","created_at":"2026-11-10T11:08:19.035Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.043Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2ea1f740a83ca3cd56952c71: start","job_status":"start","queue_duration_s":4.821926,"scheduling_latency_s":4.821957}2146gitlab # [ 563.431516] gitaly[923]: time="2026-11-10T11:08:23.784Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:33.149" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:23.770" grpc.time_ms=13.694 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2147gitlab # [ 563.445971] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.937Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2ea1f740a83ca3cd56952c71","created_at":"2026-11-10T11:08:19.035Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.043Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2ea1f740a83ca3cd56952c71: done: 0.072366 sec","job_status":"done","queue_duration_s":4.821926,"scheduling_latency_s":4.821957,"redis_calls":10,"redis_duration_s":0.003081,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000643,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005204,"duration_s":0.072366,"completed_at":"2026-11-10T11:08:23.937Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016291,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 563.492369] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.939Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"dfd1efa8fd2ec355f193d277","created_at":"2026-11-10T11:08:19.044Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.051Z","job_size_bytes":6,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-dfd1efa8fd2ec355f193d277: start","job_status":"start","queue_duration_s":4.887876,"scheduling_latency_s":4.88791}2149gitlab # [ 563.508924] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.983Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"dfd1efa8fd2ec355f193d277","created_at":"2026-11-10T11:08:19.044Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.051Z","job_size_bytes":6,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-dfd1efa8fd2ec355f193d277: done: 0.043768 sec","job_status":"done","queue_duration_s":4.887876,"scheduling_latency_s":4.88791,"redis_calls":10,"redis_duration_s":0.003714,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000726,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001992,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066694,"duration_s":0.043768,"completed_at":"2026-11-10T11:08:23.983Z","load_balancing_strategy":"primary","db_duration_s":0.004203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2150gitlab # [ 563.554397] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:23.986Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"72c347df1c25970532586fd6","created_at":"2026-11-10T11:08:19.052Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.060Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-72c347df1c25970532586fd6: start","job_status":"start","queue_duration_s":4.92592,"scheduling_latency_s":4.925957}2151gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'2152gitlab # [ 563.572347] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:24.054Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"72c347df1c25970532586fd6","created_at":"2026-11-10T11:08:19.052Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.060Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-72c347df1c25970532586fd6: done: 0.068185 sec","job_status":"done","queue_duration_s":4.92592,"scheduling_latency_s":4.925957,"redis_calls":10,"redis_duration_s":0.003291,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000817,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000703,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053614,"duration_s":0.068185,"completed_at":"2026-11-10T11:08:24.054Z","load_balancing_strategy":"primary","db_duration_s":0.011629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2153gitlab # [ 563.621862] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:24.057Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"36513fb189e6b2680c119df9","created_at":"2026-11-10T11:08:19.071Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.080Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-36513fb189e6b2680c119df9: start","job_status":"start","queue_duration_s":4.977115,"scheduling_latency_s":4.977151}2154gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.06 seconds)2155gitlab # [ 563.638888] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:24.180Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"36513fb189e6b2680c119df9","created_at":"2026-11-10T11:08:19.071Z","trace_propagation_headers":{"sentry-trace":"8444913f8ee34ed7836e121273486d4a-b9bcb0257dd04fea","baggage":"sentry-trace_id=8444913f8ee34ed7836e121273486d4a,sentry-release=Unknown"},"correlation_id":"01M72J6H0RE01RY4921EPXE27P","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.080Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-36513fb189e6b2680c119df9: done: 0.123001 sec","job_status":"done","queue_duration_s":4.977115,"scheduling_latency_s":4.977151,"redis_calls":14,"redis_duration_s":0.00482,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000873,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002022,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049236,"duration_s":0.123001,"completed_at":"2026-11-10T11:08:24.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156gitlab # [ 563.685978] gitaly[923]: time="2026-11-10T11:08:24.183Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4YECDYDRREC410PRZAZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:54.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:24.183" grpc.time_ms=0.61 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2157gitlab # [ 563.700621] nixbot[4291]: {"ts": "2026-11-10T11:08:24.199Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35584 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}2158gitlab # [ 563.703516] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:24.190Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bb08d45e075d9f5abc950d8f","created_at":"2026-11-10T11:08:19.290Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-680e9f258438476c","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"68252287530e523491ddb365","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.295Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bb08d45e075d9f5abc950d8f: start","job_status":"start","queue_duration_s":4.895529,"scheduling_latency_s":4.895573}2159gitlab # [ 563.721490] gitaly[923]: time="2026-11-10T11:08:24.246Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W diskcache=8729a71d-baf5-4844-a213-688e08b5dcba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T17:08:24.234" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:24.235" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2160gitlab # [ 563.731915] gitaly[923]: time="2026-11-10T11:08:24.246Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T17:08:24.234" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:24.235" grpc.time_ms=10.95 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161gitlab # [ 563.745951] gitaly[923]: time="2026-11-10T11:08:24.268Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA4YECDYDRREC410PRZAZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:34.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:24.257" grpc.time_ms=10.949 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2162gitlab # [ 563.759946] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:24.279Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bb08d45e075d9f5abc950d8f","created_at":"2026-11-10T11:08:19.290Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-680e9f258438476c","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"68252287530e523491ddb365","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.295Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bb08d45e075d9f5abc950d8f: done: 0.089209 sec","job_status":"done","queue_duration_s":4.895529,"scheduling_latency_s":4.895573,"gitaly_calls":1,"gitaly_duration_s":0.014819,"redis_calls":12,"redis_duration_s":0.005927,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00145,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001686,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04501612,"duration_s":0.089209,"completed_at":"2026-11-10T11:08:24.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2163gitlab # [ 563.809051] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:24.282Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a0b30979e169ce2d8b4833ba","created_at":"2026-11-10T11:08:19.500Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-680e9f258438476c","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c119dbbb698589fe17b0baea15ec45c34162566a76d4659f5799f7056b97bd20","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.522Z","job_size_bytes":467,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a0b30979e169ce2d8b4833ba: start","job_status":"start","queue_duration_s":4.759709,"scheduling_latency_s":4.759744}2164gitlab # [ 563.828469] gitaly[923]: time="2026-11-10T11:08:24.329Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:54.327" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:24.328" grpc.time_ms=0.887 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2165gitlab # [ 563.841829] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5599f99b1c2db2684ebacd4a272932f856d9fe29 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 4762166gitlab # [ 563.844553] gitaly[923]: time="2026-11-10T11:08:24.344Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T11:08:34.023" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:24.334" grpc.time_ms=10.483 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2167gitlab # [ 563.858053] gitaly[923]: time="2026-11-10T11:08:24.407Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T11:08:54.406" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:24.406" grpc.time_ms=1.256 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2168gitlab # [ 563.871351] gitaly[923]: time="2026-11-10T11:08:24.419Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=180 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T11:08:54.413" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:24.414" grpc.time_ms=4.377 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2169gitlab # [ 563.924931] gitaly[923]: time="2026-11-10T11:08:24.476Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA5DQ7E915DKYBEJHPH4W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:54.375" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:24.475" grpc.time_ms=1.027 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2170gitlab # [ 563.940437] gitaly[923]: time="2026-11-10T11:08:24.481Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:34.158" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:24.468" grpc.time_ms=11.897 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2171gitlab # [ 564.032932] gitaly[923]: time="2026-11-10T11:08:24.584Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA5DQ7E915DKYBEJHPH4W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:34.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:24.572" grpc.time_ms=11.847 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2172gitlab # [ 564.113073] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5599f99b1c2db2684ebacd4a272932f856d9fe29 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2882173gitlab # [ 564.257040] gitaly[923]: time="2026-11-10T11:08:24.809Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA5R83E5EEKYF3BSB9EX8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:24.795" grpc.time_ms=13.811 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2174gitlab # [ 564.279627] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 972175gitlab # [ 564.365553] gitaly[923]: time="2026-11-10T11:08:24.917Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA5VCTF0K435T86KXMTME grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:24.897" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2176gitlab # [ 564.376518] gitaly[923]: time="2026-11-10T11:08:24.917Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=8 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA5VCTF0K435T86KXMTME grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:24.897" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2177gitlab # [ 564.391970] gitlab-workhorse[829]: time="2026-11-10T11:08:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672178gitlab # [ 564.394472] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1052179gitlab # [ 564.396644] gitaly[923]: time="2026-11-10T11:08:24.917Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=8 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA5VCTF0K435T86KXMTME grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:24.897" grpc.time_ms=19.812 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2180gitlab # [ 564.501507] gitaly[923]: time="2026-11-10T11:08:25.053Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA602RJA8E6XXK45YAC5C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.039" grpc.time_ms=14.125 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2181gitlab # [ 564.524090] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 912182gitlab # [ 564.606482] gitaly[923]: time="2026-11-10T11:08:25.158Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA62ZSPD0D8DS09S1C03P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.144" pid=923 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc2183gitlab # [ 564.623688] gitaly[923]: time="2026-11-10T11:08:25.158Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA62ZSPD0D8DS09S1C03P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.144" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2184gitlab # [ 564.645325] gitlab-workhorse[829]: time="2026-11-10T11:08:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672185gitlab # [ 564.648114] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1032186gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'2187gitlab # [ 564.650704] gitaly[923]: time="2026-11-10T11:08:25.158Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA62ZSPD0D8DS09S1C03P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.144" grpc.time_ms=14.387 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2188gitlab # [ 564.708492] nixbot[4291]: {"ts": "2026-11-10T11:08:25.259Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35594 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}2189gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.05 seconds)2190gitlab # [ 564.753910] gitaly[923]: time="2026-11-10T11:08:25.305Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA67SDZXEF9BGWV3CSACK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.293" grpc.time_ms=12.523 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2191gitlab # [ 564.775916] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 962192gitlab # [ 564.849199] gitaly[923]: time="2026-11-10T11:08:25.401Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T11:08:55.379" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:25.380" grpc.time_ms=20.51 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2193gitlab # [ 564.874924] gitaly[923]: time="2026-11-10T11:08:25.422Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M72J9RSC4T1RDJK0R3CK0K2W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T11:08:55.405" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:25.405" grpc.time_ms=16.224 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2194gitlab # [ 564.903196] gitlab-workhorse[829]: time="2026-11-10T11:08:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162195gitlab # [ 564.905617] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1162196gitlab # [ 564.908232] sidekiq-cluster[4185]: {"severity":"WARN","time":"2026-11-10T11:08:25.447Z","class":"CreatePipelineWorker","correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"5599f99b1c2db2684ebacd4a272932f856d9fe29","after":"92d9e381fcb2582c86b3e57663b1d0593611d0fe","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"92d9e381fcb2582c86b3e57663b1d0593611d0fe","ref":"refs/heads/master"},"retry":0}2197gitlab # [ 564.915842] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.457Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a0b30979e169ce2d8b4833ba","created_at":"2026-11-10T11:08:19.500Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-680e9f258438476c","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c119dbbb698589fe17b0baea15ec45c34162566a76d4659f5799f7056b97bd20","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.522Z","job_size_bytes":467,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a0b30979e169ce2d8b4833ba: done: 1.175618 sec","job_status":"done","queue_duration_s":4.759709,"scheduling_latency_s":4.759744,"gitaly_calls":7,"gitaly_duration_s":0.099177,"redis_calls":33,"redis_duration_s":0.008449,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000774,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003999,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.055178,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.175618,"completed_at":"2026-11-10T11:08:25.457Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031027,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2198gitlab # [ 564.966813] gitaly[923]: time="2026-11-10T11:08:25.424Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6AXPPCQGJXS6C6RF8P0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.403" pid=923 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2199gitlab # [ 564.977681] nixbot[4291]: {"ts": "2026-11-10T11:08:25.507Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "41068520-5a6d-45b0-8ac3-1b051f06931c", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "92d9e381fcb2582c86b3e57663b1d0593611d0fe", "pr": null}2200gitlab # [ 564.981532] nixbot[4291]: {"ts": "2026-11-10T11:08:25.511Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35596 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}2201gitlab # [ 564.983603] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.461Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"09c8f314-3c1f-4898-a9be-f2707add0083\"}"],"class":"WebHookWorker","jid":"67d3b7f6f057b98ec2a8eafa","created_at":"2026-11-10T11:08:19.550Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-680e9f258438476c","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:12ccad5a0a7a54529a853bf492da1f964719306601e22883139ce4351c1e1023","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.555Z","job_size_bytes":1749,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-67d3b7f6f057b98ec2a8eafa: start","job_status":"start","queue_duration_s":5.905718,"scheduling_latency_s":5.905761}2202gitlab # [ 565.004450] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.528Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"09c8f314-3c1f-4898-a9be-f2707add0083\"}"],"class":"WebHookWorker","jid":"67d3b7f6f057b98ec2a8eafa","created_at":"2026-11-10T11:08:19.550Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-680e9f258438476c","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:12ccad5a0a7a54529a853bf492da1f964719306601e22883139ce4351c1e1023","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:19.555Z","job_size_bytes":1749,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-67d3b7f6f057b98ec2a8eafa: done: 0.067536 sec","job_status":"done","queue_duration_s":5.905718,"scheduling_latency_s":5.905761,"redis_calls":14,"redis_duration_s":0.004118,"redis_read_bytes":25,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.000977,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001793,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001348,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.010310979999985648,"cpu_s":0.044499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.067536,"completed_at":"2026-11-10T11:08:25.528Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203gitlab # [ 565.056604] gitaly[923]: time="2026-11-10T11:08:25.424Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1106 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6AXPPCQGJXS6C6RF8P0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.403" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2204gitlab # [ 565.071318] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.530Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"8afb4b20ed998d92580fff7c","created_at":"2026-11-10T11:08:20.888Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-af1b9afce9b14366","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:20.896Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-8afb4b20ed998d92580fff7c: start","job_status":"start","queue_duration_s":4.634162,"scheduling_latency_s":4.634197}2205gitlab # [ 565.088600] gitaly[923]: time="2026-11-10T11:08:25.425Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1106 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6AXPPCQGJXS6C6RF8P0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.403" grpc.time_ms=21.678 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2206gitlab # [ 565.104256] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.625Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"8afb4b20ed998d92580fff7c","created_at":"2026-11-10T11:08:20.888Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-af1b9afce9b14366","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:20.896Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-8afb4b20ed998d92580fff7c: done: 0.095364 sec","job_status":"done","queue_duration_s":4.634162,"scheduling_latency_s":4.634197,"redis_calls":19,"redis_duration_s":0.005135,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001168,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000747,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002129,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.014402719999999999,"duration_s":0.095364,"completed_at":"2026-11-10T11:08:25.625Z","load_balancing_strategy":"primary","db_duration_s":0.016539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207gitlab # [ 565.155510] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.628Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a749587895087140c90bba99"],"class":"WebHooks::LogExecutionWorker","jid":"21a75482a539d241e038ad3f","created_at":"2026-11-10T11:08:23.141Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-7e16e02f19ce4dd5","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:eb70bd1c9ff095d7d0a0f5b415471fbd782a8858a3cb88c15c7d11afba98bbfe","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:23.153Z","job_size_bytes":2593,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-21a75482a539d241e038ad3f: start","job_status":"start","queue_duration_s":2.475375,"scheduling_latency_s":2.475413}2208gitlab # [ 565.173738] gitaly[923]: time="2026-11-10T11:08:25.679Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA6JX53NWRZRS7CNCQN2T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.665" grpc.time_ms=13.31 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2209gitlab # [ 565.188183] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1142210gitlab # [ 565.190397] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.704Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a749587895087140c90bba99"],"class":"WebHooks::LogExecutionWorker","jid":"21a75482a539d241e038ad3f","created_at":"2026-11-10T11:08:23.141Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-7e16e02f19ce4dd5","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:eb70bd1c9ff095d7d0a0f5b415471fbd782a8858a3cb88c15c7d11afba98bbfe","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:23.153Z","job_size_bytes":2593,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-21a75482a539d241e038ad3f: done: 0.075862 sec","job_status":"done","queue_duration_s":2.475375,"scheduling_latency_s":2.475413,"redis_calls":10,"redis_duration_s":0.002539,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043804,"duration_s":0.075862,"completed_at":"2026-11-10T11:08:25.704Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011061,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2211gitlab # [ 565.241532] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:25.706Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"1f9df708bf33a5ff0e89ebe9","created_at":"2026-11-10T11:08:23.847Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-5b5d67efa7db4228","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:23.854Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1f9df708bf33a5ff0e89ebe9: start","job_status":"start","queue_duration_s":1.851571,"scheduling_latency_s":1.851605}2212gitlab # [ 565.258627] gitaly[923]: time="2026-11-10T11:08:25.790Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6PJ33W82W9MGNYK4QDG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.777" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2213gitlab # [ 565.269593] gitlab-workhorse[829]: time="2026-11-10T11:08:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672214gitlab # [ 565.271502] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1092215gitlab # [ 565.273627] gitaly[923]: time="2026-11-10T11:08:25.790Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6PJ33W82W9MGNYK4QDG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.777" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2216gitlab # [ 565.288489] gitaly[923]: time="2026-11-10T11:08:25.793Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6PJ33W82W9MGNYK4QDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:25.777" grpc.time_ms=13.254 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2217gitlab # [ 565.328453] gitaly[923]: time="2026-11-10T11:08:25.880Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:25.879" grpc.time_ms=0.727 pid=923 span.kind=server system=grpc2218gitlab # [ 565.365864] gitaly[923]: time="2026-11-10T11:08:25.917Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:25.916" grpc.time_ms=0.988 pid=923 span.kind=server system=grpc2219gitlab # [ 565.377680] gitaly[923]: time="2026-11-10T11:08:25.927Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:55.525" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:25.926" grpc.time_ms=0.829 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2220gitlab # [ 565.400146] gitaly[923]: time="2026-11-10T11:08:25.952Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:25.951" grpc.time_ms=0.553 pid=923 span.kind=server system=grpc2221gitlab # [ 565.418439] gitaly[923]: time="2026-11-10T11:08:25.958Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T11:08:55.541" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:25.941" grpc.time_ms=16.607 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2222gitlab # [ 565.434818] gitaly[923]: time="2026-11-10T11:08:25.972Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 diskcache=a1e09798-c61e-4d69-990b-87270338633e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:08:55.843" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:25.843" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2223gitlab # [ 565.445651] gitaly[923]: time="2026-11-10T11:08:25.973Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=112 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7331 command.oublock=0 command.real_time_ms=114 command.spawn_token_fork_ms=1 command.system_time_ms=57 command.user_time_ms=55 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:08:55.843" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:25.843" grpc.time_ms=129.408 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2224gitlab # [ 565.461967] gitaly[923]: time="2026-11-10T11:08:25.984Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:35.102" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:25.972" grpc.time_ms=10.972 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2225gitlab # [ 565.490534] gitaly[923]: time="2026-11-10T11:08:26.042Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T11:08:36.001" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:26.041" grpc.time_ms=0.475 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2226gitlab # [ 565.512274] gitaly[923]: time="2026-11-10T11:08:26.064Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T11:08:36.011" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:26.052" grpc.time_ms=11.379 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2227gitlab # [ 565.531657] gitaly[923]: time="2026-11-10T11:08:26.083Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T11:08:56.081" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:26.082" grpc.time_ms=1.38 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2228gitlab # [ 565.565022] gitaly[923]: time="2026-11-10T11:08:26.117Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:26.116" grpc.time_ms=0.763 pid=923 span.kind=server system=grpc2229gitlab # [ 565.594883] gitaly[923]: time="2026-11-10T11:08:26.146Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA4M0SG83G78D902TRQR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:56.144" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:26.145" grpc.time_ms=1.297 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2230gitlab # [ 565.615021] gitaly[923]: time="2026-11-10T11:08:26.151Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:26.151" grpc.time_ms=0.61 pid=923 span.kind=server system=grpc2231gitlab # [ 565.631423] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.163Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"1f9df708bf33a5ff0e89ebe9","created_at":"2026-11-10T11:08:23.847Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-5b5d67efa7db4228","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:23.854Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":4185,"job_status":"rescheduled","message":"PipelineProcessWorker JID-1f9df708bf33a5ff0e89ebe9: rescheduled"}2232gitlab # [ 565.650994] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.177Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"1f9df708bf33a5ff0e89ebe9","created_at":"2026-11-10T11:08:23.847Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-5b5d67efa7db4228","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:23.854Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1f9df708bf33a5ff0e89ebe9: done: 0.470756 sec","job_status":"done","queue_duration_s":1.851571,"scheduling_latency_s":1.851605,"gitaly_calls":3,"gitaly_duration_s":0.143825,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.026573,"redis_read_bytes":148,"redis_write_bytes":18011,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001199,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001386,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.007305,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.007633,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000588,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.004116,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003147,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.258217,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.39980504,"duration_s":0.470756,"completed_at":"2026-11-10T11:08:26.177Z","load_balancing_strategy":"primary","db_duration_s":0.061013,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2233gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe'2234gitlab # [ 565.707448] gitaly[923]: time="2026-11-10T11:08:26.170Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX diskcache=6520ba1c-06c1-4662-a7d7-c0a013f8593f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T11:08:36.027" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:26.068" pid=923 ref=refs/keep-around/92d9e381fcb2582c86b3e57663b1d0593611d0fe remote_ip="2001:db8:1::1" revision=92d9e381fcb2582c86b3e57663b1d0593611d0fe span.kind=server system=grpc username=root2235gitlab # [ 565.719569] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.180Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","67d3b7f6f057b98ec2a8eafa"],"class":"WebHooks::LogExecutionWorker","jid":"520250cf79277cc27ce75b73","created_at":"2026-11-10T11:08:25.514Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-40aea85d63e34c29","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:123b41e79b9287257588e7ca6be3923f7efcb6c20690ccb3c2761c2e3f3fe906","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.518Z","job_size_bytes":2591,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-520250cf79277cc27ce75b73: start","job_status":"start","queue_duration_s":0.661854,"scheduling_latency_s":0.661892}2236gitlab # [ 565.736448] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.229Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","67d3b7f6f057b98ec2a8eafa"],"class":"WebHooks::LogExecutionWorker","jid":"520250cf79277cc27ce75b73","created_at":"2026-11-10T11:08:25.514Z","trace_propagation_headers":{"sentry-trace":"0a9797d66e774da3a03c30507964a5cc-40aea85d63e34c29","baggage":"sentry-trace_id=0a9797d66e774da3a03c30507964a5cc,sentry-release=Unknown"},"correlation_id":"01M72J9RSC4T1RDJK0R3CK0K2W","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:123b41e79b9287257588e7ca6be3923f7efcb6c20690ccb3c2761c2e3f3fe906","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.518Z","job_size_bytes":2591,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-520250cf79277cc27ce75b73: done: 0.048608 sec","job_status":"done","queue_duration_s":0.661854,"scheduling_latency_s":0.661892,"redis_calls":7,"redis_duration_s":0.005457,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004565,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048608,"completed_at":"2026-11-10T11:08:26.228Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2237gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe', in 0.08 seconds)2238(finished: subtest: push delivers the webhook, in 122.54 seconds)2239subtest: eval and build post commit statuses2240gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2241gitlab # [ 565.783263] gitaly[923]: time="2026-11-10T11:08:26.171Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=82 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5124 command.oublock=8 command.real_time_ms=92 command.spawn_token_fork_ms=1 command.system_time_ms=44 command.user_time_ms=38 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T11:08:36.027" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T11:08:26.068" grpc.time_ms=102.287 pid=923 ref=refs/keep-around/92d9e381fcb2582c86b3e57663b1d0593611d0fe remote_ip="2001:db8:1::1" revision=92d9e381fcb2582c86b3e57663b1d0593611d0fe span.kind=server system=grpc username=root2242gitlab # [ 565.798880] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.233Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"3bfaae185107c0355ab07314","created_at":"2026-11-10T11:08:25.601Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-4c3afd3aa1b04672","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.610Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-3bfaae185107c0355ab07314: start","job_status":"start","queue_duration_s":0.622914,"scheduling_latency_s":0.622952}2243gitlab # [ 565.817617] nixbot[4291]: {"ts": "2026-11-10T11:08:26.342Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35610 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1\" 200"}2244gitlab # [ 565.820707] gitaly[923]: time="2026-11-10T11:08:26.180Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:56.077" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:26.178" grpc.time_ms=1.993 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2245gitlab # [ 565.836483] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5332246gitlab # [ 565.843333] nixbot[4291]: {"ts": "2026-11-10T11:08:26.395Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-92d9e381fcb2-6"}2247gitlab # [ 565.871319] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.422Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"3bfaae185107c0355ab07314","created_at":"2026-11-10T11:08:25.601Z","trace_propagation_headers":{"sentry-trace":"79e2b697a3b047529c223cc8a80bcb56-4c3afd3aa1b04672","baggage":"sentry-trace_id=79e2b697a3b047529c223cc8a80bcb56,sentry-release=Unknown"},"correlation_id":"01M72J6A23Z4JRKVEGMCJXQJHZ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.610Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-3bfaae185107c0355ab07314: done: 0.188779 sec","job_status":"done","queue_duration_s":0.622914,"scheduling_latency_s":0.622952,"redis_calls":11,"redis_duration_s":0.003585,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000831,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002098,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00061816,"duration_s":0.188779,"completed_at":"2026-11-10T11:08:26.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 565.928953] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.425Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"0bd47c11bcbeca7b95bec759","created_at":"2026-11-10T11:08:25.823Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.831Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-0bd47c11bcbeca7b95bec759: start","job_status":"start","queue_duration_s":0.594546,"scheduling_latency_s":0.594586}2249gitlab # [ 565.946239] gitaly[923]: time="2026-11-10T11:08:26.484Z" level=info msg="finished unary call with code OK" catfile.duration_ms=7 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=7 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA7CSW2JHR6H13CA7PHG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:56.375" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:26.476" grpc.time_ms=7.622 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2250gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.21 seconds)2251gitlab # [ 565.982649] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.533Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"0bd47c11bcbeca7b95bec759","created_at":"2026-11-10T11:08:25.823Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.831Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-0bd47c11bcbeca7b95bec759: done: 0.107941 sec","job_status":"done","queue_duration_s":0.594546,"scheduling_latency_s":0.594586,"redis_calls":7,"redis_duration_s":0.0022700000000000003,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000603,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.107941,"completed_at":"2026-11-10T11:08:26.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038755,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2252gitlab # [ 566.032810] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.542Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d1d2ca74b05a523bb36a0a4c","created_at":"2026-11-10T11:08:25.982Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.987Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d1d2ca74b05a523bb36a0a4c: start","job_status":"start","queue_duration_s":0.55454,"scheduling_latency_s":0.554598}2253gitlab # [ 566.050515] nixbot[4291]: {"ts": "2026-11-10T11:08:26.550Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-92d9e381fcb2-6", "argv0": "bwrap"}2254gitlab # [ 566.053659] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1512255gitlab # [ 566.056837] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.608Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d1d2ca74b05a523bb36a0a4c","created_at":"2026-11-10T11:08:25.982Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:25.987Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d1d2ca74b05a523bb36a0a4c: done: 0.065902 sec","job_status":"done","queue_duration_s":0.55454,"scheduling_latency_s":0.554598,"redis_calls":7,"redis_duration_s":0.002614,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000754,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065902,"completed_at":"2026-11-10T11:08:26.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010637,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256gitlab # [ 566.101832] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.611Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"285301e38c4e9cd5c934b251","created_at":"2026-11-10T11:08:26.007Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.011Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-285301e38c4e9cd5c934b251: start","job_status":"start","queue_duration_s":0.599949,"scheduling_latency_s":0.599993}2257gitlab # [ 566.127116] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.678Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"285301e38c4e9cd5c934b251","created_at":"2026-11-10T11:08:26.007Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.011Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-285301e38c4e9cd5c934b251: done: 0.067435 sec","job_status":"done","queue_duration_s":0.599949,"scheduling_latency_s":0.599993,"redis_calls":7,"redis_duration_s":0.009183,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067435,"completed_at":"2026-11-10T11:08:26.678Z","load_balancing_strategy":"primary","db_duration_s":0.009064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258gitlab # [ 566.179048] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.681Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e567d381da9b6088a29f59a2","created_at":"2026-11-10T11:08:26.028Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.032Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e567d381da9b6088a29f59a2: start","job_status":"start","queue_duration_s":0.648614,"scheduling_latency_s":0.648655}2259gitlab # [ 566.196287] nix-daemon[4471]: accepted connection from pid 5755, user nixbot2260gitlab # [ 566.197401] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.747Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e567d381da9b6088a29f59a2","created_at":"2026-11-10T11:08:26.028Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.032Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e567d381da9b6088a29f59a2: done: 0.066629 sec","job_status":"done","queue_duration_s":0.648614,"scheduling_latency_s":0.648655,"redis_calls":10,"redis_duration_s":0.00345,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001661,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000725,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066629,"completed_at":"2026-11-10T11:08:26.747Z","load_balancing_strategy":"primary","db_duration_s":0.011102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2261gitlab # [ 566.244152] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.750Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c6cf300c1447fef83069b2c3","created_at":"2026-11-10T11:08:26.035Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.039Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c6cf300c1447fef83069b2c3: start","job_status":"start","queue_duration_s":0.710716,"scheduling_latency_s":0.710755}2262gitlab # [ 566.261399] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.803Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c6cf300c1447fef83069b2c3","created_at":"2026-11-10T11:08:26.035Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.039Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c6cf300c1447fef83069b2c3: done: 0.053264 sec","job_status":"done","queue_duration_s":0.710716,"scheduling_latency_s":0.710755,"redis_calls":7,"redis_duration_s":0.003162,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053264,"completed_at":"2026-11-10T11:08:26.803Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263gitlab # [ 566.307141] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.806Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a94f2dd2bfb07149367ea2f0","created_at":"2026-11-10T11:08:26.040Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.045Z","job_size_bytes":19,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a94f2dd2bfb07149367ea2f0: start","job_status":"start","queue_duration_s":0.761045,"scheduling_latency_s":0.761085}2264gitlab # [ 566.324319] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a94f2dd2bfb07149367ea2f0","created_at":"2026-11-10T11:08:26.040Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.045Z","job_size_bytes":19,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a94f2dd2bfb07149367ea2f0: done: 0.038823 sec","job_status":"done","queue_duration_s":0.761045,"scheduling_latency_s":0.761085,"redis_calls":8,"redis_duration_s":0.002439,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001877,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038823,"completed_at":"2026-11-10T11:08:26.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005831,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265gitlab # [ 566.370147] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.846Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"107f6b11e498be1e7f4a43a1","created_at":"2026-11-10T11:08:26.046Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.050Z","job_size_bytes":66,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-107f6b11e498be1e7f4a43a1: start","job_status":"start","queue_duration_s":0.79653,"scheduling_latency_s":0.796563}2266gitlab # [ 566.388439] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.881Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"107f6b11e498be1e7f4a43a1","created_at":"2026-11-10T11:08:26.046Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.050Z","job_size_bytes":66,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-107f6b11e498be1e7f4a43a1: done: 0.034521 sec","job_status":"done","queue_duration_s":0.79653,"scheduling_latency_s":0.796563,"redis_calls":7,"redis_duration_s":0.001967,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001451,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034521,"completed_at":"2026-11-10T11:08:26.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2267gitlab # [ 566.433803] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.883Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cd278cc07359be4e0b34c4a4","created_at":"2026-11-10T11:08:26.051Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.054Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cd278cc07359be4e0b34c4a4: start","job_status":"start","queue_duration_s":0.828468,"scheduling_latency_s":0.828498}2268gitlab # [ 566.450915] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.925Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cd278cc07359be4e0b34c4a4","created_at":"2026-11-10T11:08:26.051Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.054Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cd278cc07359be4e0b34c4a4: done: 0.041831 sec","job_status":"done","queue_duration_s":0.828468,"scheduling_latency_s":0.828498,"redis_calls":7,"redis_duration_s":0.002304,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041831,"completed_at":"2026-11-10T11:08:26.925Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2269gitlab # [ 566.496799] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.927Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"363a3d836950f780fab96b54","created_at":"2026-11-10T11:08:26.056Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.059Z","job_size_bytes":6,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-363a3d836950f780fab96b54: start","job_status":"start","queue_duration_s":0.867881,"scheduling_latency_s":0.867914}2270gitlab # [ 566.513365] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"363a3d836950f780fab96b54","created_at":"2026-11-10T11:08:26.056Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.059Z","job_size_bytes":6,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-363a3d836950f780fab96b54: done: 0.02183 sec","job_status":"done","queue_duration_s":0.867881,"scheduling_latency_s":0.867914,"redis_calls":7,"redis_duration_s":0.0019010000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001378,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000523,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02183,"completed_at":"2026-11-10T11:08:26.948Z","load_balancing_strategy":"primary","db_duration_s":0.000821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2271gitlab # [ 566.557452] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.951Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e7748741658245cbb4389410","created_at":"2026-11-10T11:08:26.060Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.063Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e7748741658245cbb4389410: start","job_status":"start","queue_duration_s":0.887119,"scheduling_latency_s":0.887148}2272gitlab # [ 566.573418] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.988Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e7748741658245cbb4389410","created_at":"2026-11-10T11:08:26.060Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.063Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e7748741658245cbb4389410: done: 0.037497 sec","job_status":"done","queue_duration_s":0.887119,"scheduling_latency_s":0.887148,"redis_calls":7,"redis_duration_s":0.0020239999999999998,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037497,"completed_at":"2026-11-10T11:08:26.988Z","load_balancing_strategy":"primary","db_duration_s":0.00617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2273gitlab # [ 566.616674] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:26.990Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4a4d1611cf581446b466bf32","created_at":"2026-11-10T11:08:26.164Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.167Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4a4d1611cf581446b466bf32: start","job_status":"start","queue_duration_s":0.823043,"scheduling_latency_s":0.823079}2274gitlab # [ 566.632749] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:27.050Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4a4d1611cf581446b466bf32","created_at":"2026-11-10T11:08:26.164Z","trace_propagation_headers":{"sentry-trace":"e8b48c2eea634fb782755640f5794d36-1720eafa15604724","baggage":"sentry-trace_id=e8b48c2eea634fb782755640f5794d36,sentry-release=Unknown"},"correlation_id":"01M72JA4M0SG83G78D902TRQR5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.167Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4a4d1611cf581446b466bf32: done: 0.059466 sec","job_status":"done","queue_duration_s":0.823043,"scheduling_latency_s":0.823079,"redis_calls":8,"redis_duration_s":0.003985,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003112,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059466,"completed_at":"2026-11-10T11:08:27.050Z","load_balancing_strategy":"primary","db_duration_s":0.007798,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2275gitlab # [ 566.677912] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:27.053Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"281a3d2310aa72f1fb90dfad","created_at":"2026-11-10T11:08:26.192Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-d6ad98919ac6437d","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.198Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-281a3d2310aa72f1fb90dfad: start","job_status":"start","queue_duration_s":0.855287,"scheduling_latency_s":0.855331}2276gitlab # [ 566.698233] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:27.119Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"281a3d2310aa72f1fb90dfad","created_at":"2026-11-10T11:08:26.192Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-d6ad98919ac6437d","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.198Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-281a3d2310aa72f1fb90dfad: done: 0.065882 sec","job_status":"done","queue_duration_s":0.855287,"scheduling_latency_s":0.855331,"redis_calls":7,"redis_duration_s":0.003092,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002327,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000765,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065882,"completed_at":"2026-11-10T11:08:27.119Z","load_balancing_strategy":"primary","db_duration_s":0.008504,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2277gitlab # [ 566.742938] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:27.122Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b6279e711a552cf3f135b337","created_at":"2026-11-10T11:08:26.199Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-d6ad98919ac6437d","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.204Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b6279e711a552cf3f135b337: start","job_status":"start","queue_duration_s":0.918068,"scheduling_latency_s":0.91811}2278gitlab # [ 566.761453] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:27.297Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b6279e711a552cf3f135b337","created_at":"2026-11-10T11:08:26.199Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-d6ad98919ac6437d","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.204Z","job_size_bytes":65,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b6279e711a552cf3f135b337: done: 0.175489 sec","job_status":"done","queue_duration_s":0.918068,"scheduling_latency_s":0.91811,"redis_calls":10,"redis_duration_s":0.010822,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.004476,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.044,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.175489,"completed_at":"2026-11-10T11:08:27.297Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.065197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 566.808613] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:27.307Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8189cc82870a29d71a127e03","created_at":"2026-11-10T11:08:26.322Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-d6ad98919ac6437d","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.331Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8189cc82870a29d71a127e03: start","job_status":"start","queue_duration_s":0.976109,"scheduling_latency_s":0.976162}2280gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2281gitlab # [ 567.157247] gitaly[923]: time="2026-11-10T11:08:27.708Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA8K5SNJ5TR0WZYDM41ER grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:57.606" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:27.707" grpc.time_ms=1.005 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2282gitlab # [ 567.201828] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1302283gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.19 seconds)2284gitlab # [ 567.341247] gitaly[923]: time="2026-11-10T11:08:27.893Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T11:08:57.891" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:27.891" grpc.time_ms=1.336 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2285gitlab # [ 567.505796] gitaly[923]: time="2026-11-10T11:08:28.057Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA6VDGNRYGV465J22ZDCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:58.055" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:28.056" grpc.time_ms=1.116 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2286gitlab # [ 567.576441] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.125Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8189cc82870a29d71a127e03","created_at":"2026-11-10T11:08:26.322Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-d6ad98919ac6437d","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:26.331Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8189cc82870a29d71a127e03: done: 0.818233 sec","job_status":"done","queue_duration_s":0.976109,"scheduling_latency_s":0.976162,"gitaly_calls":2,"gitaly_duration_s":0.018474,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.273833,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.009792,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.003421,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.001071,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.01491,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001291,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.028054,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.205502,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.199,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.031,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.042,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.12,"db_ci_duration_s":0.079,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.255338,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.39578808,"duration_s":0.818233,"completed_at":"2026-11-10T11:08:28.125Z","load_balancing_strategy":"primary","db_duration_s":0.231626,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2287gitlab # [ 567.654284] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"015baca08be55c34cd07766a","created_at":"2026-11-10T11:08:27.191Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.264Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-015baca08be55c34cd07766a: start","job_status":"start","queue_duration_s":0.872442,"scheduling_latency_s":0.8725}2288gitlab # [ 567.795527] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"015baca08be55c34cd07766a","created_at":"2026-11-10T11:08:27.191Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.264Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-015baca08be55c34cd07766a: done: 0.208563 sec","job_status":"done","queue_duration_s":0.872442,"scheduling_latency_s":0.8725,"redis_calls":16,"redis_duration_s":0.051234,"redis_read_bytes":18,"redis_write_bytes":4347,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005613,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_calls":2,"redis_queues_duration_s":0.001188,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.038417,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006016,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.208563,"completed_at":"2026-11-10T11:08:28.345Z","load_balancing_strategy":"primary","db_duration_s":0.017848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2289gitlab # [ 567.857762] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dc17bc45546f2edf84d3c671","created_at":"2026-11-10T11:08:27.286Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c493778b3d9c0e5b6e3dfd5b5b990d78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dc17bc45546f2edf84d3c671: start","job_status":"start","queue_duration_s":1.045043,"scheduling_latency_s":1.04509}2290gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2291gitlab # [ 568.173582] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dc17bc45546f2edf84d3c671","created_at":"2026-11-10T11:08:27.286Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c493778b3d9c0e5b6e3dfd5b5b990d78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dc17bc45546f2edf84d3c671: done: 0.3749 sec","job_status":"done","queue_duration_s":1.045043,"scheduling_latency_s":1.04509,"redis_calls":13,"redis_duration_s":0.004421,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001286,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.312937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00116406,"exclusive_lock_hold_duration_s":0.32165606,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.3749,"completed_at":"2026-11-10T11:08:28.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.070736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2292gitlab # [ 568.245153] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7bb2d15789059958f52bd8a9","created_at":"2026-11-10T11:08:27.313Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a2073f6e5e63629a070b99a26ae5475","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.320Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7bb2d15789059958f52bd8a9: start","job_status":"start","queue_duration_s":1.408786,"scheduling_latency_s":1.408828}2293gitlab # [ 568.264552] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.744Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2294gitlab # [ 568.268936] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7bb2d15789059958f52bd8a9","created_at":"2026-11-10T11:08:27.313Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a2073f6e5e63629a070b99a26ae5475","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.320Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7bb2d15789059958f52bd8a9: done: 0.026818 sec","job_status":"done","queue_duration_s":1.408786,"scheduling_latency_s":1.408828,"redis_calls":7,"redis_duration_s":0.002412,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026818,"completed_at":"2026-11-10T11:08:28.755Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2295gitlab # [ 568.315014] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"90b1a5d0810464a14d5671ef","created_at":"2026-11-10T11:08:27.353Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d062b4409078e2a4a6661aa5f1232331","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.359Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-90b1a5d0810464a14d5671ef: start","job_status":"start","queue_duration_s":1.398955,"scheduling_latency_s":1.398989}2296gitlab # [ 568.330432] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.786Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"90b1a5d0810464a14d5671ef","created_at":"2026-11-10T11:08:27.353Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d062b4409078e2a4a6661aa5f1232331","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.359Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-90b1a5d0810464a14d5671ef: done: 0.028244 sec","job_status":"done","queue_duration_s":1.398955,"scheduling_latency_s":1.398989,"redis_calls":7,"redis_duration_s":0.002542,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000794,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028244,"completed_at":"2026-11-10T11:08:28.786Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2297gitlab # [ 568.373121] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3df5ecec2d040e8b6a9d399c","created_at":"2026-11-10T11:08:27.378Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a15ca6259600a0c844c023145849dde7","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.397Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3df5ecec2d040e8b6a9d399c: start","job_status":"start","queue_duration_s":1.39221,"scheduling_latency_s":1.392255}2298gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.21 seconds)2299gitlab # [ 568.388431] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3df5ecec2d040e8b6a9d399c","created_at":"2026-11-10T11:08:27.378Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a15ca6259600a0c844c023145849dde7","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.397Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3df5ecec2d040e8b6a9d399c: done: 0.067292 sec","job_status":"done","queue_duration_s":1.39221,"scheduling_latency_s":1.392255,"redis_calls":10,"redis_duration_s":0.003869,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002238,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01959404,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.067292,"completed_at":"2026-11-10T11:08:28.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015013,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2300gitlab # [ 568.433394] gitaly[923]: time="2026-11-10T11:08:28.904Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JA9R0C0HWFSXCJA00H1QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:08:58.499" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:28.899" grpc.time_ms=4.038 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2301gitlab # [ 568.447343] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1622302gitlab # [ 568.450303] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b669abb2fcaca9b6019f21bc","created_at":"2026-11-10T11:08:27.434Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6aaeb9038634d9f29aac16d0f4ed5e8d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.441Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b669abb2fcaca9b6019f21bc: start","job_status":"start","queue_duration_s":1.417718,"scheduling_latency_s":1.41776}2303gitlab # [ 568.465803] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b669abb2fcaca9b6019f21bc","created_at":"2026-11-10T11:08:27.434Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6aaeb9038634d9f29aac16d0f4ed5e8d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.441Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b669abb2fcaca9b6019f21bc: done: 0.086141 sec","job_status":"done","queue_duration_s":1.417718,"scheduling_latency_s":1.41776,"redis_calls":9,"redis_duration_s":0.004761,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002479,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00154436,"exclusive_lock_hold_duration_s":0.0302311,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.086141,"completed_at":"2026-11-10T11:08:28.945Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2304gitlab # [ 568.512706] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2d8fc75b1e07e5eb8e6b46fd","created_at":"2026-11-10T11:08:27.459Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ead83c2d7faa54172d16f0d82657606","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.465Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2d8fc75b1e07e5eb8e6b46fd: start","job_status":"start","queue_duration_s":1.483817,"scheduling_latency_s":1.483861}2305gitlab # [ 568.526919] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2d8fc75b1e07e5eb8e6b46fd","created_at":"2026-11-10T11:08:27.459Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ead83c2d7faa54172d16f0d82657606","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.465Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2d8fc75b1e07e5eb8e6b46fd: done: 0.036148 sec","job_status":"done","queue_duration_s":1.483817,"scheduling_latency_s":1.483861,"redis_calls":9,"redis_duration_s":0.003789,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001683,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00109638,"exclusive_lock_hold_duration_s":0.00663276,"duration_s":0.036148,"completed_at":"2026-11-10T11:08:28.985Z","load_balancing_strategy":"primary","db_duration_s":0.001862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2306gitlab # [ 568.568993] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:28.988Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c24474487e132378291cb14f","created_at":"2026-11-10T11:08:27.678Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93dfe58b83a0693ef4f5094d79fd7b19","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.684Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c24474487e132378291cb14f: start","job_status":"start","queue_duration_s":1.304474,"scheduling_latency_s":1.304507}2307gitlab # [ 568.584584] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c24474487e132378291cb14f","created_at":"2026-11-10T11:08:27.678Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93dfe58b83a0693ef4f5094d79fd7b19","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.684Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c24474487e132378291cb14f: done: 0.030165 sec","job_status":"done","queue_duration_s":1.304474,"scheduling_latency_s":1.304507,"redis_calls":9,"redis_duration_s":0.0029230000000000003,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00139,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00100022,"exclusive_lock_hold_duration_s":0.00529944,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.030165,"completed_at":"2026-11-10T11:08:29.018Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2308gitlab # [ 568.628455] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"23c95874cbb49c7f8438f9d7","created_at":"2026-11-10T11:08:27.713Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c643a0726b0c188e96fc27686696701","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:08:27.720Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-23c95874cbb49c7f8438f9d7: start","job_status":"start","queue_duration_s":1.30089,"scheduling_latency_s":1.300922}2309gitlab # [ 568.642555] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.033Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2310gitlab # [ 568.646225] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"23c95874cbb49c7f8438f9d7","created_at":"2026-11-10T11:08:27.713Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c643a0726b0c188e96fc27686696701","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:08:27.720Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-23c95874cbb49c7f8438f9d7: done: 0.021302 sec","job_status":"done","queue_duration_s":1.30089,"scheduling_latency_s":1.300922,"redis_calls":7,"redis_duration_s":0.001915,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021302,"completed_at":"2026-11-10T11:08:29.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2311gitlab # [ 568.688182] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.044Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8ffcf5198c31e7d7f2629f52","created_at":"2026-11-10T11:08:27.729Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6931a092f52a6557b023af3c9cf9e0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.735Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8ffcf5198c31e7d7f2629f52: start","job_status":"start","queue_duration_s":1.309219,"scheduling_latency_s":1.309249}2312gitlab # [ 568.704993] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8ffcf5198c31e7d7f2629f52","created_at":"2026-11-10T11:08:27.729Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6931a092f52a6557b023af3c9cf9e0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.735Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8ffcf5198c31e7d7f2629f52: done: 0.02647 sec","job_status":"done","queue_duration_s":1.309219,"scheduling_latency_s":1.309249,"redis_calls":7,"redis_duration_s":0.0019060000000000001,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001379,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02647,"completed_at":"2026-11-10T11:08:29.071Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2313gitlab # [ 568.747747] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fbb69f280bd1f7a4fb9d7ced","created_at":"2026-11-10T11:08:27.770Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85b65fe744da36ac0c28ac9fd9cb1cd2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:08:27.780Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fbb69f280bd1f7a4fb9d7ced: start","job_status":"start","queue_duration_s":1.292945,"scheduling_latency_s":1.292982}2314gitlab # [ 568.762077] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.085Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2315gitlab # [ 568.766034] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fbb69f280bd1f7a4fb9d7ced","created_at":"2026-11-10T11:08:27.770Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85b65fe744da36ac0c28ac9fd9cb1cd2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:08:27.780Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fbb69f280bd1f7a4fb9d7ced: done: 0.020702 sec","job_status":"done","queue_duration_s":1.292945,"scheduling_latency_s":1.292982,"redis_calls":7,"redis_duration_s":0.0018579999999999998,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000542,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020702,"completed_at":"2026-11-10T11:08:29.093Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2316gitlab # [ 568.807898] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"daa838f76bdaace2ef811b27","created_at":"2026-11-10T11:08:27.808Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1892e3c6033f2bb30e8c3462e2a3edc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.818Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-daa838f76bdaace2ef811b27: start","job_status":"start","queue_duration_s":1.27709,"scheduling_latency_s":1.277121}2317gitlab # [ 568.823588] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.107Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2318gitlab # [ 568.827384] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"daa838f76bdaace2ef811b27","created_at":"2026-11-10T11:08:27.808Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1892e3c6033f2bb30e8c3462e2a3edc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.818Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-daa838f76bdaace2ef811b27: done: 0.019847 sec","job_status":"done","queue_duration_s":1.27709,"scheduling_latency_s":1.277121,"redis_calls":7,"redis_duration_s":0.00175,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000533,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019847,"completed_at":"2026-11-10T11:08:29.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2319gitlab # [ 568.877304] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"580b9603545427e0563dae25","created_at":"2026-11-10T11:08:27.815Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-229edb0677964daf","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.825Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-580b9603545427e0563dae25: start","job_status":"start","queue_duration_s":1.292811,"scheduling_latency_s":1.292839}2320gitlab # [ 568.894598] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.179Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"580b9603545427e0563dae25","created_at":"2026-11-10T11:08:27.815Z","trace_propagation_headers":{"sentry-trace":"dd347f6127424a2f92321d3b1f6dd72b-229edb0677964daf","baggage":"sentry-trace_id=dd347f6127424a2f92321d3b1f6dd72b,sentry-release=Unknown"},"correlation_id":"01M72JA6VDGNRYGV465J22ZDCX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.825Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-580b9603545427e0563dae25: done: 0.061658 sec","job_status":"done","queue_duration_s":1.292811,"scheduling_latency_s":1.292839,"redis_calls":7,"redis_duration_s":0.002218,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001609,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000609,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061658,"completed_at":"2026-11-10T11:08:29.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011981,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321gitlab # [ 568.945271] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.182Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"25ee111c38ce244b3e21a233","created_at":"2026-11-10T11:08:27.841Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"898efdd2efe98e0d9abf173a909e3191","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.865Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-25ee111c38ce244b3e21a233: start","job_status":"start","queue_duration_s":1.317273,"scheduling_latency_s":1.317308}2322gitlab # [ 568.960297] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.239Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"25ee111c38ce244b3e21a233","created_at":"2026-11-10T11:08:27.841Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"898efdd2efe98e0d9abf173a909e3191","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.865Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-25ee111c38ce244b3e21a233: done: 0.056943 sec","job_status":"done","queue_duration_s":1.317273,"scheduling_latency_s":1.317308,"redis_calls":9,"redis_duration_s":0.003091,"redis_read_bytes":7,"redis_write_bytes":1108,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000561,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001912,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056943,"completed_at":"2026-11-10T11:08:29.239Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2323gitlab # [ 569.004167] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ad50305912c1866e51154db4","created_at":"2026-11-10T11:08:27.880Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbdf8944350c7b4bcae01c536659be53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:08:27.894Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ad50305912c1866e51154db4: start","job_status":"start","queue_duration_s":1.34708,"scheduling_latency_s":1.347114}2324gitlab # [ 569.017890] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ad50305912c1866e51154db4","created_at":"2026-11-10T11:08:27.880Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbdf8944350c7b4bcae01c536659be53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:08:27.894Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ad50305912c1866e51154db4: done: 0.031842 sec","job_status":"done","queue_duration_s":1.34708,"scheduling_latency_s":1.347114,"redis_calls":7,"redis_duration_s":0.00195,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001418,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031842,"completed_at":"2026-11-10T11:08:29.273Z","load_balancing_strategy":"primary","db_duration_s":0.002517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2325gitlab # [ 569.059335] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2e771e9122eec21fc0aab7aa","created_at":"2026-11-10T11:08:27.917Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"377de3c9be770f8519253a177d42c4ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.937Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2e771e9122eec21fc0aab7aa: start","job_status":"start","queue_duration_s":1.3371,"scheduling_latency_s":1.33713}2326gitlab # [ 569.075050] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2e771e9122eec21fc0aab7aa","created_at":"2026-11-10T11:08:27.917Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"377de3c9be770f8519253a177d42c4ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.937Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2e771e9122eec21fc0aab7aa: done: 0.020191 sec","job_status":"done","queue_duration_s":1.3371,"scheduling_latency_s":1.33713,"redis_calls":7,"redis_duration_s":0.0017460000000000002,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020191,"completed_at":"2026-11-10T11:08:29.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2327gitlab # [ 569.121682] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"076ad1a154e9ced3999a40ab","created_at":"2026-11-10T11:08:27.955Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"627852bb1a10527b7c188d031b5e8ca9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.969Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-076ad1a154e9ced3999a40ab: start","job_status":"start","queue_duration_s":1.327464,"scheduling_latency_s":1.327494}2328gitlab # [ 569.137354] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.322Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"076ad1a154e9ced3999a40ab","created_at":"2026-11-10T11:08:27.955Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"627852bb1a10527b7c188d031b5e8ca9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:27.969Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-076ad1a154e9ced3999a40ab: done: 0.024898 sec","job_status":"done","queue_duration_s":1.327464,"scheduling_latency_s":1.327494,"redis_calls":9,"redis_duration_s":0.002513,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001196,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00085254,"exclusive_lock_hold_duration_s":0.0041295,"duration_s":0.024898,"completed_at":"2026-11-10T11:08:29.322Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2329gitlab # [ 569.180538] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.324Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"deefe0d14a1299e3dd2dead8","created_at":"2026-11-10T11:08:27.996Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1bd24ea2456a73c99e3ce0138deaf40","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.007Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-deefe0d14a1299e3dd2dead8: start","job_status":"start","queue_duration_s":1.31657,"scheduling_latency_s":1.316599}2330gitlab # [ 569.220900] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"deefe0d14a1299e3dd2dead8","created_at":"2026-11-10T11:08:27.996Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1bd24ea2456a73c99e3ce0138deaf40","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.007Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-deefe0d14a1299e3dd2dead8: done: 0.447691 sec","job_status":"done","queue_duration_s":1.31657,"scheduling_latency_s":1.316599,"redis_calls":562,"redis_duration_s":0.108152,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.10651,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.396797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.447691,"completed_at":"2026-11-10T11:08:29.771Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2331gitlab # [ 569.263319] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cd268143ab2bddd38e4ee82d","created_at":"2026-11-10T11:08:28.056Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bd189e1134997645383482d10590441","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.064Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cd268143ab2bddd38e4ee82d: start","job_status":"start","queue_duration_s":1.710557,"scheduling_latency_s":1.710597}2332gitlab # [ 569.278276] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cd268143ab2bddd38e4ee82d","created_at":"2026-11-10T11:08:28.056Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bd189e1134997645383482d10590441","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.064Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cd268143ab2bddd38e4ee82d: done: 0.033614 sec","job_status":"done","queue_duration_s":1.710557,"scheduling_latency_s":1.710597,"redis_calls":7,"redis_duration_s":0.002286,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000624,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033614,"completed_at":"2026-11-10T11:08:29.808Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2333gitlab # [ 569.321076] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.811Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7140531d1319eb8bd869637c","created_at":"2026-11-10T11:08:28.087Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13670a5234906933716c4b50175138ff","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.096Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7140531d1319eb8bd869637c: start","job_status":"start","queue_duration_s":1.715246,"scheduling_latency_s":1.715289}2334gitlab # [ 569.335088] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.837Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7140531d1319eb8bd869637c","created_at":"2026-11-10T11:08:28.087Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13670a5234906933716c4b50175138ff","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.096Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7140531d1319eb8bd869637c: done: 0.026111 sec","job_status":"done","queue_duration_s":1.715246,"scheduling_latency_s":1.715289,"redis_calls":9,"redis_duration_s":0.0025259999999999996,"redis_read_bytes":6,"redis_write_bytes":789,"redis_cache_calls":2,"redis_cache_duration_s":0.000502,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026111,"completed_at":"2026-11-10T11:08:29.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2335gitlab # [ 569.378091] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f962d062e1760bb776526f17","created_at":"2026-11-10T11:08:28.118Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab5df569db557f225d09588a1437b2d8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.130Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f962d062e1760bb776526f17: start","job_status":"start","queue_duration_s":1.708902,"scheduling_latency_s":1.708938}2336gitlab # [ 569.393384] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.853Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2337gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2338gitlab # [ 569.398996] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f962d062e1760bb776526f17","created_at":"2026-11-10T11:08:28.118Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab5df569db557f225d09588a1437b2d8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.130Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f962d062e1760bb776526f17: done: 0.022743 sec","job_status":"done","queue_duration_s":1.708902,"scheduling_latency_s":1.708938,"redis_calls":7,"redis_duration_s":0.002053,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022743,"completed_at":"2026-11-10T11:08:29.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2339gitlab # [ 569.440694] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.864Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4822d4a7bf78f8c316bfd974","created_at":"2026-11-10T11:08:28.146Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0580d0a8c44e4ebc472bc57dd3d552a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.153Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4822d4a7bf78f8c316bfd974: start","job_status":"start","queue_duration_s":1.711529,"scheduling_latency_s":1.711561}2340gitlab # [ 569.456870] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.920Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4822d4a7bf78f8c316bfd974","created_at":"2026-11-10T11:08:28.146Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0580d0a8c44e4ebc472bc57dd3d552a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.153Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4822d4a7bf78f8c316bfd974: done: 0.056136 sec","job_status":"done","queue_duration_s":1.711529,"scheduling_latency_s":1.711561,"redis_calls":9,"redis_duration_s":0.0030039999999999997,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001514,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00149,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00092204,"exclusive_lock_hold_duration_s":0.03060134,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.056136,"completed_at":"2026-11-10T11:08:29.920Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2341gitlab # [ 569.502871] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2191267ca6ae103671d8ad08","created_at":"2026-11-10T11:08:28.161Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"843d87d8f2fe1fcba314396fe3e536cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.167Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2191267ca6ae103671d8ad08: start","job_status":"start","queue_duration_s":1.756083,"scheduling_latency_s":1.756117}2342gitlab # [ 569.518088] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.933Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2343gitlab # [ 569.521504] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2191267ca6ae103671d8ad08","created_at":"2026-11-10T11:08:28.161Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"843d87d8f2fe1fcba314396fe3e536cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.167Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2191267ca6ae103671d8ad08: done: 0.017778 sec","job_status":"done","queue_duration_s":1.756083,"scheduling_latency_s":1.756117,"redis_calls":7,"redis_duration_s":0.001839,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001426,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017778,"completed_at":"2026-11-10T11:08:29.940Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2344gitlab # [ 569.565057] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:29.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3bb646bd2d5d9fc40c59e94d","created_at":"2026-11-10T11:08:28.198Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.214Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3bb646bd2d5d9fc40c59e94d: start","job_status":"start","queue_duration_s":1.728507,"scheduling_latency_s":1.728535}2345gitlab # [ 569.579369] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3bb646bd2d5d9fc40c59e94d","created_at":"2026-11-10T11:08:28.198Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.214Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3bb646bd2d5d9fc40c59e94d: done: 0.138417 sec","job_status":"done","queue_duration_s":1.728507,"scheduling_latency_s":1.728535,"redis_calls":60,"redis_duration_s":0.019624,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000963,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.016867,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.123134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.138417,"completed_at":"2026-11-10T11:08:30.080Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2346gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.21 seconds)2347gitlab # [ 569.623255] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.084Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"59a541ed1c222140f9572684","created_at":"2026-11-10T11:08:28.235Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fce31dfa768b1db5749511a99e90ac62","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.249Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-59a541ed1c222140f9572684: start","job_status":"start","queue_duration_s":1.834985,"scheduling_latency_s":1.835024}2348gitlab # [ 569.639614] gitaly[923]: time="2026-11-10T11:08:30.124Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAAYDV26WDF11Y7F1SQV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:00.022" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:30.122" grpc.time_ms=0.868 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2349gitlab # [ 569.653594] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1452350gitlab # [ 569.656349] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.121Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"59a541ed1c222140f9572684","created_at":"2026-11-10T11:08:28.235Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fce31dfa768b1db5749511a99e90ac62","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.249Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-59a541ed1c222140f9572684: done: 0.03712 sec","job_status":"done","queue_duration_s":1.834985,"scheduling_latency_s":1.835024,"redis_calls":7,"redis_duration_s":0.002775,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000594,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03712,"completed_at":"2026-11-10T11:08:30.121Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006991,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2351gitlab # [ 569.698589] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.125Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"007b414b11d10dc922aa80bf","created_at":"2026-11-10T11:08:28.283Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f5bd4d1a61ee9b81e499ea0f8338608","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.294Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-007b414b11d10dc922aa80bf: start","job_status":"start","queue_duration_s":1.831187,"scheduling_latency_s":1.83124}2352gitlab # [ 569.713458] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"007b414b11d10dc922aa80bf","created_at":"2026-11-10T11:08:28.283Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-cbe6cc829a3c4995","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f5bd4d1a61ee9b81e499ea0f8338608","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.294Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-007b414b11d10dc922aa80bf: done: 0.04245 sec","job_status":"done","queue_duration_s":1.831187,"scheduling_latency_s":1.83124,"redis_calls":7,"redis_duration_s":0.002964,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000819,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.04245,"completed_at":"2026-11-10T11:08:30.168Z","load_balancing_strategy":"primary","db_duration_s":0.010931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2353gitlab # [ 569.761362] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.170Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7e9989faca0396ab52ab2f25","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7e9989faca0396ab52ab2f25: start","job_status":"start","queue_duration_s":1.865656,"scheduling_latency_s":1.865689}2354gitlab # [ 569.774352] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.191Z","message":"No existing merge request to be cleaned up."}2355gitlab # [ 569.776158] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.207Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7e9989faca0396ab52ab2f25","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7e9989faca0396ab52ab2f25: done: 0.037005 sec","job_status":"done","queue_duration_s":1.865656,"scheduling_latency_s":1.865689,"redis_calls":10,"redis_duration_s":0.003843,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001398,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002445,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037005,"completed_at":"2026-11-10T11:08:30.207Z","load_balancing_strategy":"primary","db_duration_s":0.003389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2356gitlab # [ 569.816668] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.210Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"75f082dc45f43fc1015d6426","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-75f082dc45f43fc1015d6426: start","job_status":"start","queue_duration_s":1.905368,"scheduling_latency_s":1.905406}2357gitlab # [ 569.829615] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.229Z","message":"No existing merge request to be cleaned up."}2358gitlab # [ 569.831379] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.244Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"75f082dc45f43fc1015d6426","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-75f082dc45f43fc1015d6426: done: 0.034746 sec","job_status":"done","queue_duration_s":1.905368,"scheduling_latency_s":1.905406,"redis_calls":10,"redis_duration_s":0.003248,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001689,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034746,"completed_at":"2026-11-10T11:08:30.244Z","load_balancing_strategy":"primary","db_duration_s":0.003147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2359gitlab # [ 569.872369] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.247Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"60cd8a6e5a514114cca20471","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-60cd8a6e5a514114cca20471: start","job_status":"start","queue_duration_s":1.942286,"scheduling_latency_s":1.94232}2360gitlab # [ 569.885231] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.265Z","message":"No existing merge request to be cleaned up."}2361gitlab # [ 569.886946] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.280Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"60cd8a6e5a514114cca20471","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-60cd8a6e5a514114cca20471: done: 0.033484 sec","job_status":"done","queue_duration_s":1.942286,"scheduling_latency_s":1.94232,"redis_calls":10,"redis_duration_s":0.002895,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001655,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033484,"completed_at":"2026-11-10T11:08:30.280Z","load_balancing_strategy":"primary","db_duration_s":0.003279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2362gitlab # [ 569.927699] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.282Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"68b01f44dcfc31c41d594cf3","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-68b01f44dcfc31c41d594cf3: start","job_status":"start","queue_duration_s":1.978119,"scheduling_latency_s":1.978153}2363gitlab # [ 569.940885] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.301Z","message":"No existing merge request to be cleaned up."}2364gitlab # [ 569.942600] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.314Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"68b01f44dcfc31c41d594cf3","created_at":"2026-11-10T11:08:28.261Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-2607bcb6772445ae","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ec9e1b63381e6ee86340878d3554f712","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:28.304Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-68b01f44dcfc31c41d594cf3: done: 0.031938 sec","job_status":"done","queue_duration_s":1.978119,"scheduling_latency_s":1.978153,"redis_calls":10,"redis_duration_s":0.002704,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031938,"completed_at":"2026-11-10T11:08:30.314Z","load_balancing_strategy":"primary","db_duration_s":0.002936,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2365gitlab # [ 569.984334] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea543c0b5b69733a126feee3","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea543c0b5b69733a126feee3: start","job_status":"start","queue_duration_s":0.249204,"scheduling_latency_s":0.249232}2366gitlab # [ 569.997179] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea543c0b5b69733a126feee3","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea543c0b5b69733a126feee3: done: 0.018364 sec","job_status":"done","queue_duration_s":0.249204,"scheduling_latency_s":0.249232,"redis_calls":13,"redis_duration_s":0.0025719999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018364,"completed_at":"2026-11-10T11:08:30.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2367gitlab # [ 570.038282] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf000b380690cbc2a08de831","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf000b380690cbc2a08de831: start","job_status":"start","queue_duration_s":0.269172,"scheduling_latency_s":0.269199}2368gitlab # [ 570.051288] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf000b380690cbc2a08de831","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf000b380690cbc2a08de831: done: 0.018256 sec","job_status":"done","queue_duration_s":0.269172,"scheduling_latency_s":0.269199,"redis_calls":13,"redis_duration_s":0.002487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001615,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018256,"completed_at":"2026-11-10T11:08:30.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2369gitlab # [ 570.092735] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d545b7af1b8e6a0b735693c4","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d545b7af1b8e6a0b735693c4: start","job_status":"start","queue_duration_s":0.288996,"scheduling_latency_s":0.289021}2370gitlab # [ 570.105804] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d545b7af1b8e6a0b735693c4","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d545b7af1b8e6a0b735693c4: done: 0.018549 sec","job_status":"done","queue_duration_s":0.288996,"scheduling_latency_s":0.289021,"redis_calls":13,"redis_duration_s":0.002465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018549,"completed_at":"2026-11-10T11:08:30.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2371gitlab # [ 570.146949] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f861214de92f6c565b0052a1","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f861214de92f6c565b0052a1: start","job_status":"start","queue_duration_s":0.309204,"scheduling_latency_s":0.30923}2372gitlab # [ 570.161202] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f861214de92f6c565b0052a1","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f861214de92f6c565b0052a1: done: 0.018929 sec","job_status":"done","queue_duration_s":0.309204,"scheduling_latency_s":0.30923,"redis_calls":13,"redis_duration_s":0.002522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001643,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018929,"completed_at":"2026-11-10T11:08:30.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2373gitlab # [ 570.202149] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94600cea2243652618f7e754","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94600cea2243652618f7e754: start","job_status":"start","queue_duration_s":0.329802,"scheduling_latency_s":0.329826}2374gitlab # [ 570.214877] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94600cea2243652618f7e754","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94600cea2243652618f7e754: done: 0.1389 sec","job_status":"done","queue_duration_s":0.329802,"scheduling_latency_s":0.329826,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001758,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.131272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.1389,"completed_at":"2026-11-10T11:08:30.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2375gitlab # [ 570.256449] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc9e12bb4d4769ee8999a93c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc9e12bb4d4769ee8999a93c: start","job_status":"start","queue_duration_s":0.471441,"scheduling_latency_s":0.471473}2376gitlab # [ 570.269364] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc9e12bb4d4769ee8999a93c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc9e12bb4d4769ee8999a93c: done: 0.021703 sec","job_status":"done","queue_duration_s":0.471441,"scheduling_latency_s":0.471473,"redis_calls":13,"redis_duration_s":0.002592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001696,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021703,"completed_at":"2026-11-10T11:08:30.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2377gitlab # [ 570.310723] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7f9000e0efd4b24a4fd1ae6","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7f9000e0efd4b24a4fd1ae6: start","job_status":"start","queue_duration_s":0.495013,"scheduling_latency_s":0.495039}2378gitlab # [ 570.324065] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7f9000e0efd4b24a4fd1ae6","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7f9000e0efd4b24a4fd1ae6: done: 0.020536 sec","job_status":"done","queue_duration_s":0.495013,"scheduling_latency_s":0.495039,"redis_calls":13,"redis_duration_s":0.002552,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020536,"completed_at":"2026-11-10T11:08:30.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2379gitlab # [ 570.366625] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d882ef82b4fdde7b6af7dd88","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d882ef82b4fdde7b6af7dd88: start","job_status":"start","queue_duration_s":0.517409,"scheduling_latency_s":0.517439}2380gitlab # [ 570.379889] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d882ef82b4fdde7b6af7dd88","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d882ef82b4fdde7b6af7dd88: done: 0.038452 sec","job_status":"done","queue_duration_s":0.517409,"scheduling_latency_s":0.517439,"redis_calls":13,"redis_duration_s":0.006006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038452,"completed_at":"2026-11-10T11:08:30.622Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2381gitlab # [ 570.422885] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea97239de82050013e8193e8","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea97239de82050013e8193e8: start","job_status":"start","queue_duration_s":0.55959,"scheduling_latency_s":0.559634}2382gitlab # [ 570.436265] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea97239de82050013e8193e8","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea97239de82050013e8193e8: done: 0.035874 sec","job_status":"done","queue_duration_s":0.55959,"scheduling_latency_s":0.559634,"redis_calls":13,"redis_duration_s":0.004561,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001775,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002786,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035874,"completed_at":"2026-11-10T11:08:30.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2383gitlab # [ 570.477258] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"041cbd85d2fd48c3ac4e05f1","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-041cbd85d2fd48c3ac4e05f1: start","job_status":"start","queue_duration_s":0.599853,"scheduling_latency_s":0.599896}2384gitlab # [ 570.490325] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"041cbd85d2fd48c3ac4e05f1","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-041cbd85d2fd48c3ac4e05f1: done: 0.031296 sec","job_status":"done","queue_duration_s":0.599853,"scheduling_latency_s":0.599896,"redis_calls":13,"redis_duration_s":0.0032329999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031296,"completed_at":"2026-11-10T11:08:30.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2385gitlab # [ 570.535390] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0be61367759e7456519bf47","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0be61367759e7456519bf47: start","job_status":"start","queue_duration_s":0.633509,"scheduling_latency_s":0.633547}2386gitlab # [ 570.549084] nix-daemon[4471]: accepted connection from pid 5787, user nixbot2387gitlab # [ 570.550082] nix-daemon[4471]: accepted connection from pid 5755, user nixbot2388gitlab # [ 570.551078] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0be61367759e7456519bf47","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0be61367759e7456519bf47: done: 0.036196 sec","job_status":"done","queue_duration_s":0.633509,"scheduling_latency_s":0.633547,"redis_calls":13,"redis_duration_s":0.004143,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002504,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036196,"completed_at":"2026-11-10T11:08:30.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2389gitlab # [ 570.593261] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09b35e4c7c7483ffee3dd1a7","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09b35e4c7c7483ffee3dd1a7: start","job_status":"start","queue_duration_s":0.67219,"scheduling_latency_s":0.672231}2390gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2391gitlab # [ 570.606468] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09b35e4c7c7483ffee3dd1a7","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09b35e4c7c7483ffee3dd1a7: done: 0.042407 sec","job_status":"done","queue_duration_s":0.67219,"scheduling_latency_s":0.672231,"redis_calls":13,"redis_duration_s":0.012403999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010469,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042407,"completed_at":"2026-11-10T11:08:30.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2392gitlab # [ 570.648392] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02d8754d38a6401c76c84699","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02d8754d38a6401c76c84699: start","job_status":"start","queue_duration_s":0.716585,"scheduling_latency_s":0.716616}2393gitlab # [ 570.661631] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02d8754d38a6401c76c84699","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02d8754d38a6401c76c84699: done: 0.035238 sec","job_status":"done","queue_duration_s":0.716585,"scheduling_latency_s":0.716616,"redis_calls":13,"redis_duration_s":0.004431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002555,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035238,"completed_at":"2026-11-10T11:08:30.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2394gitlab # [ 570.703007] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55d0b0a91beeeaa44994905c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55d0b0a91beeeaa44994905c: start","job_status":"start","queue_duration_s":0.754315,"scheduling_latency_s":0.754352}2395gitlab # [ 570.716315] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55d0b0a91beeeaa44994905c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55d0b0a91beeeaa44994905c: done: 0.032476 sec","job_status":"done","queue_duration_s":0.754315,"scheduling_latency_s":0.754352,"redis_calls":13,"redis_duration_s":0.003291,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032476,"completed_at":"2026-11-10T11:08:30.853Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2396gitlab # [ 570.757458] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29feb00bc61f049c77a1f5a0","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29feb00bc61f049c77a1f5a0: start","job_status":"start","queue_duration_s":0.790362,"scheduling_latency_s":0.790401}2397gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.16 seconds)2398gitlab # [ 570.770559] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29feb00bc61f049c77a1f5a0","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29feb00bc61f049c77a1f5a0: done: 0.029417 sec","job_status":"done","queue_duration_s":0.790362,"scheduling_latency_s":0.790401,"redis_calls":13,"redis_duration_s":0.002992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029417,"completed_at":"2026-11-10T11:08:30.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2399gitlab # [ 570.813002] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cadfc3017b5f16eff1985719","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cadfc3017b5f16eff1985719: start","job_status":"start","queue_duration_s":0.821852,"scheduling_latency_s":0.821884}2400gitlab # [ 570.826511] gitaly[923]: time="2026-11-10T11:08:31.304Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAC4A0QRB0SJQAC17QTTW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:01.203" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:31.303" grpc.time_ms=0.733 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2401gitlab # [ 570.840360] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 972402gitlab # [ 570.843088] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cadfc3017b5f16eff1985719","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cadfc3017b5f16eff1985719: done: 0.026258 sec","job_status":"done","queue_duration_s":0.821852,"scheduling_latency_s":0.821884,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00107,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026258,"completed_at":"2026-11-10T11:08:30.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2403gitlab # [ 570.885093] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f6fde08208b39b21b73227b","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f6fde08208b39b21b73227b: start","job_status":"start","queue_duration_s":0.851354,"scheduling_latency_s":0.851401}2404gitlab # [ 570.898141] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f6fde08208b39b21b73227b","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f6fde08208b39b21b73227b: done: 0.019766 sec","job_status":"done","queue_duration_s":0.851354,"scheduling_latency_s":0.851401,"redis_calls":13,"redis_duration_s":0.0020469999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001313,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019766,"completed_at":"2026-11-10T11:08:30.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2405gitlab # [ 570.939872] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcb6aa50b180848327dcfcfc","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcb6aa50b180848327dcfcfc: start","job_status":"start","queue_duration_s":0.872647,"scheduling_latency_s":0.872674}2406gitlab # [ 570.953375] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcb6aa50b180848327dcfcfc","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcb6aa50b180848327dcfcfc: done: 0.022354 sec","job_status":"done","queue_duration_s":0.872647,"scheduling_latency_s":0.872674,"redis_calls":13,"redis_duration_s":0.002781,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001784,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022354,"completed_at":"2026-11-10T11:08:30.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2407gitlab # [ 570.994749] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a41dc553399af283e1945e26","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a41dc553399af283e1945e26: start","job_status":"start","queue_duration_s":0.898457,"scheduling_latency_s":0.898507}2408gitlab # [ 571.008168] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a41dc553399af283e1945e26","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a41dc553399af283e1945e26: done: 0.025704 sec","job_status":"done","queue_duration_s":0.898457,"scheduling_latency_s":0.898507,"redis_calls":13,"redis_duration_s":0.002915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025704,"completed_at":"2026-11-10T11:08:30.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2409gitlab # [ 571.050553] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:30.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01bf2e560c09420bede99cbd","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01bf2e560c09420bede99cbd: start","job_status":"start","queue_duration_s":0.926105,"scheduling_latency_s":0.926135}2410gitlab # [ 571.063937] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01bf2e560c09420bede99cbd","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01bf2e560c09420bede99cbd: done: 0.02258 sec","job_status":"done","queue_duration_s":0.926105,"scheduling_latency_s":0.926135,"redis_calls":13,"redis_duration_s":0.00256,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02258,"completed_at":"2026-11-10T11:08:31.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2411gitlab # [ 571.105651] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5840ec8aa46ca79186a0281f","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5840ec8aa46ca79186a0281f: start","job_status":"start","queue_duration_s":0.951611,"scheduling_latency_s":0.95164}2412gitlab # [ 571.118802] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5840ec8aa46ca79186a0281f","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5840ec8aa46ca79186a0281f: done: 0.026113 sec","job_status":"done","queue_duration_s":0.951611,"scheduling_latency_s":0.95164,"redis_calls":13,"redis_duration_s":0.005298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002169,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026113,"completed_at":"2026-11-10T11:08:31.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2413gitlab # [ 571.160143] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70b1280e41309be7cea1fb91","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70b1280e41309be7cea1fb91: start","job_status":"start","queue_duration_s":0.980106,"scheduling_latency_s":0.980138}2414gitlab # [ 571.172974] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70b1280e41309be7cea1fb91","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70b1280e41309be7cea1fb91: done: 0.025338 sec","job_status":"done","queue_duration_s":0.980106,"scheduling_latency_s":0.980138,"redis_calls":13,"redis_duration_s":0.003256,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025338,"completed_at":"2026-11-10T11:08:31.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2415gitlab # [ 571.214141] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcc51d0accc0acfa9b7799ef","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcc51d0accc0acfa9b7799ef: start","job_status":"start","queue_duration_s":1.00719,"scheduling_latency_s":1.007221}2416gitlab # [ 571.228450] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcc51d0accc0acfa9b7799ef","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcc51d0accc0acfa9b7799ef: done: 0.022514 sec","job_status":"done","queue_duration_s":1.00719,"scheduling_latency_s":1.007221,"redis_calls":13,"redis_duration_s":0.002513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022514,"completed_at":"2026-11-10T11:08:31.096Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2417gitlab # [ 571.269463] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01d79448673f914318ceeb4c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01d79448673f914318ceeb4c: start","job_status":"start","queue_duration_s":1.033006,"scheduling_latency_s":1.033042}2418gitlab # [ 571.282554] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01d79448673f914318ceeb4c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01d79448673f914318ceeb4c: done: 0.026786 sec","job_status":"done","queue_duration_s":1.033006,"scheduling_latency_s":1.033042,"redis_calls":13,"redis_duration_s":0.003098,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001342,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026786,"completed_at":"2026-11-10T11:08:31.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2419gitlab # [ 571.323906] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f456f7b85e45b623c33c4164","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f456f7b85e45b623c33c4164: start","job_status":"start","queue_duration_s":1.062616,"scheduling_latency_s":1.062654}2420gitlab # [ 571.337166] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f456f7b85e45b623c33c4164","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f456f7b85e45b623c33c4164: done: 0.03422 sec","job_status":"done","queue_duration_s":1.062616,"scheduling_latency_s":1.062654,"redis_calls":13,"redis_duration_s":0.0037790000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001315,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002464,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03422,"completed_at":"2026-11-10T11:08:31.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2421gitlab # [ 571.380062] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c04b446d348f2b96b4cb1ed","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c04b446d348f2b96b4cb1ed: start","job_status":"start","queue_duration_s":1.099362,"scheduling_latency_s":1.0994}2422gitlab # [ 571.392954] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c04b446d348f2b96b4cb1ed","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c04b446d348f2b96b4cb1ed: done: 0.035279 sec","job_status":"done","queue_duration_s":1.099362,"scheduling_latency_s":1.0994,"redis_calls":13,"redis_duration_s":0.004615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035279,"completed_at":"2026-11-10T11:08:31.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2423gitlab # [ 571.435301] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"749c8d75c4f2ee61d168e467","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-749c8d75c4f2ee61d168e467: start","job_status":"start","queue_duration_s":1.137296,"scheduling_latency_s":1.137335}2424gitlab # [ 571.448679] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"749c8d75c4f2ee61d168e467","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-749c8d75c4f2ee61d168e467: done: 0.037435 sec","job_status":"done","queue_duration_s":1.137296,"scheduling_latency_s":1.137335,"redis_calls":13,"redis_duration_s":0.005355,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003788,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037435,"completed_at":"2026-11-10T11:08:31.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2425gitlab # [ 571.489776] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d02539516b19cd17489e710","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d02539516b19cd17489e710: start","job_status":"start","queue_duration_s":1.180392,"scheduling_latency_s":1.180435}2426gitlab # [ 571.502815] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d02539516b19cd17489e710","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d02539516b19cd17489e710: done: 0.035036 sec","job_status":"done","queue_duration_s":1.180392,"scheduling_latency_s":1.180435,"redis_calls":13,"redis_duration_s":0.003482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001215,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035036,"completed_at":"2026-11-10T11:08:31.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2427gitlab # [ 571.544419] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07f2420c118cc37743666da1","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07f2420c118cc37743666da1: start","job_status":"start","queue_duration_s":1.217772,"scheduling_latency_s":1.21781}2428gitlab # [ 571.557434] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07f2420c118cc37743666da1","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07f2420c118cc37743666da1: done: 0.029729 sec","job_status":"done","queue_duration_s":1.217772,"scheduling_latency_s":1.21781,"redis_calls":13,"redis_duration_s":0.003157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029729,"completed_at":"2026-11-10T11:08:31.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2429gitlab # [ 571.599505] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1e18dec1fbbb287236fa090","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1e18dec1fbbb287236fa090: start","job_status":"start","queue_duration_s":1.250659,"scheduling_latency_s":1.250697}2430gitlab # [ 571.613075] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1e18dec1fbbb287236fa090","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1e18dec1fbbb287236fa090: done: 0.028282 sec","job_status":"done","queue_duration_s":1.250659,"scheduling_latency_s":1.250697,"redis_calls":13,"redis_duration_s":0.003617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028282,"completed_at":"2026-11-10T11:08:31.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2431gitlab # [ 571.656411] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"303fb7754fb90de7a2c12956","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-303fb7754fb90de7a2c12956: start","job_status":"start","queue_duration_s":1.280977,"scheduling_latency_s":1.281011}2432gitlab # [ 571.669577] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"303fb7754fb90de7a2c12956","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-303fb7754fb90de7a2c12956: done: 0.026221 sec","job_status":"done","queue_duration_s":1.280977,"scheduling_latency_s":1.281011,"redis_calls":13,"redis_duration_s":0.0028060000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001817,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026221,"completed_at":"2026-11-10T11:08:31.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2433gitlab # [ 571.711277] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42ac0d193b950d336fe833bd","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42ac0d193b950d336fe833bd: start","job_status":"start","queue_duration_s":1.309198,"scheduling_latency_s":1.309232}2434gitlab # [ 571.724689] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42ac0d193b950d336fe833bd","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42ac0d193b950d336fe833bd: done: 0.026269 sec","job_status":"done","queue_duration_s":1.309198,"scheduling_latency_s":1.309232,"redis_calls":13,"redis_duration_s":0.002909,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026269,"completed_at":"2026-11-10T11:08:31.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2435gitlab # [ 571.765886] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b55cc2c235a3067aab2fca34","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b55cc2c235a3067aab2fca34: start","job_status":"start","queue_duration_s":1.337663,"scheduling_latency_s":1.337695}2436gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2437gitlab # [ 571.778998] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b55cc2c235a3067aab2fca34","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b55cc2c235a3067aab2fca34: done: 0.027511 sec","job_status":"done","queue_duration_s":1.337663,"scheduling_latency_s":1.337695,"redis_calls":13,"redis_duration_s":0.003414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027511,"completed_at":"2026-11-10T11:08:31.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2438gitlab # [ 571.819321] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3891b7c4929e7ba68cdcc9b6","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3891b7c4929e7ba68cdcc9b6: start","job_status":"start","queue_duration_s":1.367308,"scheduling_latency_s":1.367341}2439gitlab # [ 571.832467] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3891b7c4929e7ba68cdcc9b6","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3891b7c4929e7ba68cdcc9b6: done: 0.027748 sec","job_status":"done","queue_duration_s":1.367308,"scheduling_latency_s":1.367341,"redis_calls":13,"redis_duration_s":0.0035510000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002426,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027748,"completed_at":"2026-11-10T11:08:31.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2440gitlab # [ 571.874719] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8af67dcca197acc9b6d47fab","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8af67dcca197acc9b6d47fab: start","job_status":"start","queue_duration_s":1.398063,"scheduling_latency_s":1.3981}2441gitlab # [ 571.887711] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8af67dcca197acc9b6d47fab","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8af67dcca197acc9b6d47fab: done: 0.025975 sec","job_status":"done","queue_duration_s":1.398063,"scheduling_latency_s":1.3981,"redis_calls":13,"redis_duration_s":0.002922,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025975,"completed_at":"2026-11-10T11:08:31.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2442gitlab # [ 571.929000] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22325d0ad8d3eab7a60e1b09","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22325d0ad8d3eab7a60e1b09: start","job_status":"start","queue_duration_s":1.426275,"scheduling_latency_s":1.426306}2443gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.18 seconds)2444gitlab # [ 571.941939] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22325d0ad8d3eab7a60e1b09","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22325d0ad8d3eab7a60e1b09: done: 0.0255 sec","job_status":"done","queue_duration_s":1.426275,"scheduling_latency_s":1.426306,"redis_calls":13,"redis_duration_s":0.003006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0255,"completed_at":"2026-11-10T11:08:31.518Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2445gitlab # [ 571.982952] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5548c1ec540f2ad77dabfc76","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5548c1ec540f2ad77dabfc76: start","job_status":"start","queue_duration_s":1.453881,"scheduling_latency_s":1.453914}2446gitlab # [ 571.996576] gitaly[923]: time="2026-11-10T11:08:32.480Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAD8G53ZGSMNX6MSA91AG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:02.378" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:32.479" grpc.time_ms=0.849 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2447gitlab # [ 572.010329] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1202448gitlab # [ 572.013161] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5548c1ec540f2ad77dabfc76","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5548c1ec540f2ad77dabfc76: done: 0.0219 sec","job_status":"done","queue_duration_s":1.453881,"scheduling_latency_s":1.453914,"redis_calls":13,"redis_duration_s":0.002449,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001558,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0219,"completed_at":"2026-11-10T11:08:31.542Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2449gitlab # [ 572.054194] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"770f913024d3ec209a767ec2","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-770f913024d3ec209a767ec2: start","job_status":"start","queue_duration_s":1.477357,"scheduling_latency_s":1.477384}2450gitlab # [ 572.067367] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"770f913024d3ec209a767ec2","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-770f913024d3ec209a767ec2: done: 0.019499 sec","job_status":"done","queue_duration_s":1.477357,"scheduling_latency_s":1.477384,"redis_calls":13,"redis_duration_s":0.002858,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019499,"completed_at":"2026-11-10T11:08:31.563Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2451gitlab # [ 572.109648] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07082568e8f9fd01f0a123cb","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07082568e8f9fd01f0a123cb: start","job_status":"start","queue_duration_s":1.498706,"scheduling_latency_s":1.498735}2452gitlab # [ 572.122690] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07082568e8f9fd01f0a123cb","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07082568e8f9fd01f0a123cb: done: 0.020007 sec","job_status":"done","queue_duration_s":1.498706,"scheduling_latency_s":1.498735,"redis_calls":13,"redis_duration_s":0.002439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001549,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020007,"completed_at":"2026-11-10T11:08:31.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2453gitlab # [ 572.164049] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3bdf76487d9cb8a84147ac","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3bdf76487d9cb8a84147ac: start","job_status":"start","queue_duration_s":1.521497,"scheduling_latency_s":1.521526}2454gitlab # [ 572.176930] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3bdf76487d9cb8a84147ac","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3bdf76487d9cb8a84147ac: done: 0.020302 sec","job_status":"done","queue_duration_s":1.521497,"scheduling_latency_s":1.521526,"redis_calls":13,"redis_duration_s":0.002646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001683,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020302,"completed_at":"2026-11-10T11:08:31.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2455gitlab # [ 572.218077] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93dae94797405f5d5fdd7e97","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93dae94797405f5d5fdd7e97: start","job_status":"start","queue_duration_s":1.543485,"scheduling_latency_s":1.543513}2456gitlab # [ 572.231150] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93dae94797405f5d5fdd7e97","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93dae94797405f5d5fdd7e97: done: 0.021134 sec","job_status":"done","queue_duration_s":1.543485,"scheduling_latency_s":1.543513,"redis_calls":13,"redis_duration_s":0.002497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021134,"completed_at":"2026-11-10T11:08:31.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2457gitlab # [ 572.277578] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed6474f30ecfd93cb882b96f","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed6474f30ecfd93cb882b96f: start","job_status":"start","queue_duration_s":1.566428,"scheduling_latency_s":1.566456}2458gitlab # [ 572.291864] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed6474f30ecfd93cb882b96f","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed6474f30ecfd93cb882b96f: done: 0.019801 sec","job_status":"done","queue_duration_s":1.566428,"scheduling_latency_s":1.566456,"redis_calls":13,"redis_duration_s":0.0023959999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001519,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019801,"completed_at":"2026-11-10T11:08:31.653Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2459gitlab # [ 572.333784] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2d7ce8817cc3e74dd166dd4","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2d7ce8817cc3e74dd166dd4: start","job_status":"start","queue_duration_s":1.587991,"scheduling_latency_s":1.58802}2460gitlab # [ 572.346882] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2d7ce8817cc3e74dd166dd4","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2d7ce8817cc3e74dd166dd4: done: 0.020515 sec","job_status":"done","queue_duration_s":1.587991,"scheduling_latency_s":1.58802,"redis_calls":13,"redis_duration_s":0.002678,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020515,"completed_at":"2026-11-10T11:08:31.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2461gitlab # [ 572.389918] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99f46f5fb8b5b9b656a16a6","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99f46f5fb8b5b9b656a16a6: start","job_status":"start","queue_duration_s":1.610438,"scheduling_latency_s":1.610466}2462gitlab # [ 572.402999] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99f46f5fb8b5b9b656a16a6","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99f46f5fb8b5b9b656a16a6: done: 0.020149 sec","job_status":"done","queue_duration_s":1.610438,"scheduling_latency_s":1.610466,"redis_calls":13,"redis_duration_s":0.00249,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001518,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020149,"completed_at":"2026-11-10T11:08:31.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2463gitlab # [ 572.444470] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2ece9fdb0d63d2a6ca867c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2ece9fdb0d63d2a6ca867c: start","job_status":"start","queue_duration_s":1.632251,"scheduling_latency_s":1.632279}2464gitlab # [ 572.457489] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2ece9fdb0d63d2a6ca867c","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2ece9fdb0d63d2a6ca867c: done: 0.044169 sec","job_status":"done","queue_duration_s":1.632251,"scheduling_latency_s":1.632279,"redis_calls":13,"redis_duration_s":0.017548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044169,"completed_at":"2026-11-10T11:08:31.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2465gitlab # [ 572.500228] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889456eca49c91ee8f89ae64","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889456eca49c91ee8f89ae64: start","job_status":"start","queue_duration_s":1.679329,"scheduling_latency_s":1.679367}2466gitlab # [ 572.513103] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889456eca49c91ee8f89ae64","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889456eca49c91ee8f89ae64: done: 0.030058 sec","job_status":"done","queue_duration_s":1.679329,"scheduling_latency_s":1.679367,"redis_calls":13,"redis_duration_s":0.003464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00224,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030058,"completed_at":"2026-11-10T11:08:31.776Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2467gitlab # [ 572.554263] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22866df4172be60a5512c98b","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22866df4172be60a5512c98b: start","job_status":"start","queue_duration_s":1.712855,"scheduling_latency_s":1.712929}2468gitlab # [ 572.567386] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22866df4172be60a5512c98b","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22866df4172be60a5512c98b: done: 0.033352 sec","job_status":"done","queue_duration_s":1.712855,"scheduling_latency_s":1.712929,"redis_calls":13,"redis_duration_s":0.003717,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002327,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033352,"completed_at":"2026-11-10T11:08:31.813Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2469gitlab # [ 572.608810] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b2f02e6d80f74a53cfaf015","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b2f02e6d80f74a53cfaf015: start","job_status":"start","queue_duration_s":1.748519,"scheduling_latency_s":1.748556}2470gitlab # [ 572.621771] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b2f02e6d80f74a53cfaf015","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b2f02e6d80f74a53cfaf015: done: 0.028081 sec","job_status":"done","queue_duration_s":1.748519,"scheduling_latency_s":1.748556,"redis_calls":13,"redis_duration_s":0.003033,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00106,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028081,"completed_at":"2026-11-10T11:08:31.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2471gitlab # [ 572.663525] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3795e32a32fdeb7828fc463a","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3795e32a32fdeb7828fc463a: start","job_status":"start","queue_duration_s":1.778821,"scheduling_latency_s":1.778856}2472gitlab # [ 572.676858] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3795e32a32fdeb7828fc463a","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3795e32a32fdeb7828fc463a: done: 0.026522 sec","job_status":"done","queue_duration_s":1.778821,"scheduling_latency_s":1.778856,"redis_calls":13,"redis_duration_s":0.003215,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026522,"completed_at":"2026-11-10T11:08:31.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2473gitlab # [ 572.719250] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93095f2a7ecd6885a6e30e6e","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93095f2a7ecd6885a6e30e6e: start","job_status":"start","queue_duration_s":1.807336,"scheduling_latency_s":1.807368}2474gitlab # [ 572.732700] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:31.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93095f2a7ecd6885a6e30e6e","created_at":"2026-11-10T11:08:29.953Z","trace_propagation_headers":{"sentry-trace":"77dda326710d4dabbc81fac4f422ba94-6b1f903dab874457","baggage":"sentry-trace_id=77dda326710d4dabbc81fac4f422ba94,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db30c669fc1a71e5c25851da671fc2f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:08:30.067Z","job_size_bytes":2,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93095f2a7ecd6885a6e30e6e: done: 0.025822 sec","job_status":"done","queue_duration_s":1.807336,"scheduling_latency_s":1.807368,"redis_calls":13,"redis_duration_s":0.003192,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025822,"completed_at":"2026-11-10T11:08:31.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2475gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2476gitlab # [ 573.113837] gitaly[923]: time="2026-11-10T11:08:33.665Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAEDBTDR5AQP76VR8SS0C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:03.563" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:33.663" grpc.time_ms=1.242 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2477gitlab # [ 573.157614] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1292478gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.18 seconds)2479gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2480gitlab # [ 574.290949] gitaly[923]: time="2026-11-10T11:08:34.842Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAFJ9BHXWNGWNJGV52SZC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:04.440" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:34.841" grpc.time_ms=0.971 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2481gitlab # [ 574.339920] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1292482gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.17 seconds)2483gitlab # [ 575.328727] nix-daemon[4471]: accepted connection from pid 5755, user nixbot2484gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2485gitlab # [ 575.457480] gitaly[923]: time="2026-11-10T11:08:36.008Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGPYH606DMSG676HWDW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:06.006" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:36.007" grpc.time_ms=1.006 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2486gitlab # [ 575.478803] gitaly[923]: time="2026-11-10T11:08:36.020Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:06.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:36.019" grpc.time_ms=0.892 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2487gitlab # [ 575.524501] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1412488gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.18 seconds)2489gitlab # [ 575.536438] gitaly[923]: time="2026-11-10T11:08:36.088Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:46.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.075" grpc.time_ms=13.211 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2490gitlab # [ 575.598935] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.149Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6a33b2a03e2fcf1b594d1542","created_at":"2026-11-10T11:08:36.137Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-1927cc5fe1bd4a2f","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.146Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6a33b2a03e2fcf1b594d1542: start","job_status":"start","queue_duration_s":0.002846,"scheduling_latency_s":0.002899}2491gitlab # [ 575.625732] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2342492gitlab # [ 575.715151] gitaly[923]: time="2026-11-10T11:08:36.267Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGYP97V56N29DM2K9D05 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:06.265" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:36.265" grpc.time_ms=0.976 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2493gitlab # [ 575.782967] gitaly[923]: time="2026-11-10T11:08:36.335Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:36.334" grpc.time_ms=0.659 pid=923 span.kind=server system=grpc2494gitlab # [ 575.800829] gitaly[923]: time="2026-11-10T11:08:36.352Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JAGYP97V56N29DM2K9D05 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:46.087" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.337" grpc.time_ms=15.342 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2495gitlab # [ 575.827998] gitaly[923]: time="2026-11-10T11:08:36.379Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:36.379" grpc.time_ms=0.742 pid=923 span.kind=server system=grpc2496gitlab # [ 575.883925] gitaly[923]: time="2026-11-10T11:08:36.435Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T11:08:36.435" grpc.time_ms=0.623 pid=923 span.kind=server system=grpc2497gitlab # [ 575.901057] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2702498gitlab # [ 575.905602] gitaly[923]: time="2026-11-10T11:08:36.454Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:09:06.297" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.297" operations="[repack_refs]" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2499gitlab # [ 575.923673] gitaly[923]: time="2026-11-10T11:08:36.458Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:09:06.297" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.297" pid=923 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-11-10 11:06:09.58855584 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2500gitlab # [ 575.937001] gitaly[923]: time="2026-11-10T11:08:36.460Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 diskcache=5bc66ed5-97d5-4c61-b1f2-70d35dc71b96 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:09:06.297" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.297" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2501gitlab # [ 575.955943] gitaly[923]: time="2026-11-10T11:08:36.460Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=126 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7355 command.oublock=0 command.real_time_ms=155 command.spawn_token_fork_ms=1 command.system_time_ms=64 command.user_time_ms=62 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:09:06.297" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.297" grpc.time_ms=162.971 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2502gitlab # [ 575.970697] gitaly[923]: time="2026-11-10T11:08:36.487Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:09:06.297" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.297" pid=923 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2503gitlab # [ 575.985980] gitaly[923]: time="2026-11-10T11:08:36.488Z" level=info msg="optimized repository" clean_stale_data_time=29.0753ms clean_work_trees_time="197.98µs" commit_graph_time="1.3µs" correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="2.54µs" pid=923 prune_time=280ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="2.04µs" storage_name=default system=housekeeping total_time=29.34466ms2504gitlab # [ 575.992426] gitaly[923]: time="2026-11-10T11:08:36.488Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T11:09:06.297" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.297" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2505gitlab # [ 576.017910] gitaly[923]: time="2026-11-10T11:08:36.569Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAH7T0TPZWGC90Q4R7XD2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:06.568" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:36.568" grpc.time_ms=0.917 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2506gitlab # [ 576.031791] gitaly[923]: time="2026-11-10T11:08:36.577Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T11:09:06.571" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:36.572" grpc.time_ms=4.948 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2507gitlab # [ 576.079437] gitaly[923]: time="2026-11-10T11:08:36.631Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAGQ0NGD1QKVAAR5BSS07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:06.629" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:36.629" grpc.time_ms=1.327 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2508gitlab # [ 576.096214] gitaly[923]: time="2026-11-10T11:08:36.644Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JAH7T0TPZWGC90Q4R7XD2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T11:08:46.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T11:08:36.630" grpc.time_ms=13.675 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2509gitlab # [ 576.110579] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.651Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d287b43b3b0d4983d803207a","created_at":"2026-11-10T11:08:36.648Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"6a33b2a03e2fcf1b594d1542","job_size_bytes":3,"pid":4185,"job_status":"deduplicated","message":"PipelineProcessWorker JID-d287b43b3b0d4983d803207a: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2510gitlab # [ 576.127113] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.654Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6a33b2a03e2fcf1b594d1542","created_at":"2026-11-10T11:08:36.137Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-1927cc5fe1bd4a2f","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.146Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":4185,"job_status":"rescheduled","message":"PipelineProcessWorker JID-6a33b2a03e2fcf1b594d1542: rescheduled"}2511gitlab # [ 576.143146] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.665Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6a33b2a03e2fcf1b594d1542","created_at":"2026-11-10T11:08:36.137Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-1927cc5fe1bd4a2f","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.146Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6a33b2a03e2fcf1b594d1542: done: 0.516408 sec","job_status":"done","queue_duration_s":0.002846,"scheduling_latency_s":0.002899,"gitaly_calls":3,"gitaly_duration_s":0.18246,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.022878,"redis_read_bytes":239,"redis_write_bytes":21782,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001808,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00189,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.005412,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14467,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.008246,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000469,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000554,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002691,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.016,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.268929,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.43732326,"duration_s":0.516408,"completed_at":"2026-11-10T11:08:36.665Z","load_balancing_strategy":"primary","db_duration_s":0.064852,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2512gitlab # [ 576.199482] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"87b0e7e19e88964019fe025e","created_at":"2026-11-10T11:08:36.281Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.286Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-87b0e7e19e88964019fe025e: start","job_status":"start","queue_duration_s":0.381584,"scheduling_latency_s":0.381628}2513gitlab # [ 576.217815] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"87b0e7e19e88964019fe025e","created_at":"2026-11-10T11:08:36.281Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.286Z","job_size_bytes":23,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-87b0e7e19e88964019fe025e: done: 0.084091 sec","job_status":"done","queue_duration_s":0.381584,"scheduling_latency_s":0.381628,"redis_calls":7,"redis_duration_s":0.0020670000000000003,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001461,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000606,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.084091,"completed_at":"2026-11-10T11:08:36.752Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021344,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2514gitlab # [ 576.264192] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.754Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"14914779276c27dd26e2598f","created_at":"2026-11-10T11:08:36.478Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.482Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-14914779276c27dd26e2598f: start","job_status":"start","queue_duration_s":0.271739,"scheduling_latency_s":0.271774}2515gitlab # [ 576.281813] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/92d9e381fcb2582c86b3e57663b1d0593611d0fe HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2782516gitlab # [ 576.284530] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.816Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"14914779276c27dd26e2598f","created_at":"2026-11-10T11:08:36.478Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.482Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-14914779276c27dd26e2598f: done: 0.062349 sec","job_status":"done","queue_duration_s":0.271739,"scheduling_latency_s":0.271774,"redis_calls":7,"redis_duration_s":0.002909,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002183,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000726,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062349,"completed_at":"2026-11-10T11:08:36.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2517gitlab # [ 576.329600] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.819Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"60af81ef5589dcf180c869e5","created_at":"2026-11-10T11:08:36.495Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.499Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-60af81ef5589dcf180c869e5: start","job_status":"start","queue_duration_s":0.320461,"scheduling_latency_s":0.320521}2518gitlab # [ 576.346456] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"60af81ef5589dcf180c869e5","created_at":"2026-11-10T11:08:36.495Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.499Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-60af81ef5589dcf180c869e5: done: 0.058331 sec","job_status":"done","queue_duration_s":0.320461,"scheduling_latency_s":0.320521,"redis_calls":7,"redis_duration_s":0.002233,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0016,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058331,"completed_at":"2026-11-10T11:08:36.878Z","load_balancing_strategy":"primary","db_duration_s":0.008113,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2519gitlab # [ 576.390779] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.880Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"b69e305bf4fed87d568eb1f6","created_at":"2026-11-10T11:08:36.511Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.515Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b69e305bf4fed87d568eb1f6: start","job_status":"start","queue_duration_s":0.364716,"scheduling_latency_s":0.364755}2520gitlab # [ 576.408194] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.941Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"b69e305bf4fed87d568eb1f6","created_at":"2026-11-10T11:08:36.511Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.515Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b69e305bf4fed87d568eb1f6: done: 0.061242 sec","job_status":"done","queue_duration_s":0.364716,"scheduling_latency_s":0.364755,"redis_calls":10,"redis_duration_s":0.002882,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000649,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061242,"completed_at":"2026-11-10T11:08:36.941Z","load_balancing_strategy":"primary","db_duration_s":0.009022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2521gitlab # [ 576.455332] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.944Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"29a46b42a12a5feffa19f8f0","created_at":"2026-11-10T11:08:36.517Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.520Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-29a46b42a12a5feffa19f8f0: start","job_status":"start","queue_duration_s":0.42349,"scheduling_latency_s":0.423562}2522gitlab # [ 576.473104] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:36.999Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"29a46b42a12a5feffa19f8f0","created_at":"2026-11-10T11:08:36.517Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.520Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-29a46b42a12a5feffa19f8f0: done: 0.055025 sec","job_status":"done","queue_duration_s":0.42349,"scheduling_latency_s":0.423562,"redis_calls":7,"redis_duration_s":0.002403,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055025,"completed_at":"2026-11-10T11:08:36.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2523gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses2524gitlab # [ 576.522554] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.002Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"764a07961fff50baa96a9baa","created_at":"2026-11-10T11:08:36.521Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.525Z","job_size_bytes":19,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-764a07961fff50baa96a9baa: start","job_status":"start","queue_duration_s":0.476272,"scheduling_latency_s":0.476312}2525gitlab # [ 576.539959] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.044Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"764a07961fff50baa96a9baa","created_at":"2026-11-10T11:08:36.521Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.525Z","job_size_bytes":19,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-764a07961fff50baa96a9baa: done: 0.042483 sec","job_status":"done","queue_duration_s":0.476272,"scheduling_latency_s":0.476312,"redis_calls":8,"redis_duration_s":0.002241,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001518,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042483,"completed_at":"2026-11-10T11:08:37.044Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2526gitlab # [ 576.585669] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.046Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0fcd2c791156101c684b4e19","created_at":"2026-11-10T11:08:36.527Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.530Z","job_size_bytes":66,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0fcd2c791156101c684b4e19: start","job_status":"start","queue_duration_s":0.515221,"scheduling_latency_s":0.515255}2527gitlab # [ 576.604634] nixbot[4291]: {"ts": "2026-11-10T11:08:37.067Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "92d9e381fcb2582c86b3e57663b1d0593611d0fe", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}2528gitlab # [ 576.608487] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.081Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0fcd2c791156101c684b4e19","created_at":"2026-11-10T11:08:36.527Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.530Z","job_size_bytes":66,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0fcd2c791156101c684b4e19: done: 0.035059 sec","job_status":"done","queue_duration_s":0.515221,"scheduling_latency_s":0.515255,"redis_calls":7,"redis_duration_s":0.0019290000000000002,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035059,"completed_at":"2026-11-10T11:08:37.081Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2529gitlab # [ 576.655310] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.083Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d99ade4ff2d32f106dc9d184","created_at":"2026-11-10T11:08:36.532Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.536Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d99ade4ff2d32f106dc9d184: start","job_status":"start","queue_duration_s":0.547371,"scheduling_latency_s":0.547402}2530gitlab # [ 576.679024] gitaly[923]: time="2026-11-10T11:08:37.230Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72JAHW7CNAXYMSQMGF13DXF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T11:08:37.219" grpc.time_ms=10.682 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc2531gitlab # [ 576.693631] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1032532gitlab # [ 576.695869] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.240Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d99ade4ff2d32f106dc9d184","created_at":"2026-11-10T11:08:36.532Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.536Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d99ade4ff2d32f106dc9d184: done: 0.156522 sec","job_status":"done","queue_duration_s":0.547371,"scheduling_latency_s":0.547402,"redis_calls":7,"redis_duration_s":0.005092,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004534,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.156522,"completed_at":"2026-11-10T11:08:37.240Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2533gitlab # [ 576.741282] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.242Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"bbfe87654cd26de60b345d58","created_at":"2026-11-10T11:08:36.544Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.547Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-bbfe87654cd26de60b345d58: start","job_status":"start","queue_duration_s":0.695288,"scheduling_latency_s":0.695324}2534gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses, in 0.23 seconds)2535(finished: subtest: eval and build post commit statuses, in 10.97 seconds)2536(finished: run the VM test script, in 779.06 seconds)2537gitlab # [ 576.758693] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.289Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"bbfe87654cd26de60b345d58","created_at":"2026-11-10T11:08:36.544Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.547Z","job_size_bytes":3,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-bbfe87654cd26de60b345d58: done: 0.047327 sec","job_status":"done","queue_duration_s":0.695288,"scheduling_latency_s":0.695324,"redis_calls":7,"redis_duration_s":0.0019060000000000001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047327,"completed_at":"2026-11-10T11:08:37.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2538gitlab # [ 576.811619] sidekiq-cluster[4185]: {"severity":"INFO","time":"2026-11-10T11:08:37.292Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0ca90b74f40da29fcec2e880","created_at":"2026-11-10T11:08:36.547Z","trace_propagation_headers":{"sentry-trace":"dd1b476dc4ad4a6c811b7d9bd1e77371-d7d1def4728c4661","baggage":"sentry-trace_id=dd1b476dc4ad4a6c811b7d9bd1e77371,sentry-release=Unknown"},"correlation_id":"01M72JAGQ0NGD1QKVAAR5BSS07","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T11:08:36.550Z","job_size_bytes":6,"pid":4185,"sidekiq_tid":"m629","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0ca90b74f40da29fcec2e880: start","job_status":"start","queue_duration_s":0.741704,"scheduling_latency_s":0.74175}2539gitlab # [ 576.829803] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/92d9e381fcb2582c86b3e57663b1d0593611d0fe/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1482540gitlab # [ 576.835297] gitlab-workhorse[829]: time="2026-11-10T11:08:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672541gitlab # [ 576.838506] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/11/10:11:08:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1032542test script finished in 779.15s2543gitlab # [ 576.840590] gitaly[923]: time="2026-11-10T11:08:37.255Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72JAHX7AJ35M6MYVV8FZ28G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T11:09:07.152" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T11:08:37.252" grpc.time_ms=2.709 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2544cleanup2545kill QemuMachine (pid 46)2546gitlab # [ 576.854524] gitaly[923]: time="2026-11-10T11:08:37.335Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72JAHZHZ1MZHKX980M44TTQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2547(finished: cleanup, in 5.51 seconds)2548additionally exposed symbols:2549 gitlab,2550 vlan1,2551 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2552[]2553[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-10T11:06:23.072Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2554[]2555[]2556[]2557[]2558[]2559[]2560[]2561[]2562[]2563[]2564[]2565[]2566[]2567[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-10T11:08:25.825343Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2568[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2569[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2570[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2571[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2572[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2573[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2574[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2575[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2576[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2577[{'id': 5, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-11-10T11:08:26.303Z', 'started_at': None, 'finished_at': '2026-11-10T11:08:36.122Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '92d9e381fcb2582c86b3e57663b1d0593611d0fe', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-11-10T11:08:36.405Z', 'started_at': None, 'finished_at': '2026-11-10T11:08:36.692Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]